YaCy Release 0.99

0.99

Major Changes   
Jump to: Bugfixes / Other Changes

CommitDescription
Wed Nov 03 23:52:03 CET 2010
by apfelmaennchen
a update to ymarks (please test if you wish):
- import HTML (e.g. FF export) via /api/ymarks/import.html
- view your import via /api/ymarks/test.html
- get a xml list via /api/ymarks/get_ymark_list.xml?tags=&folders=
- delete bookmark tables via standard interface /Tables_p.html
it is still very experimental!! 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7299 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/api/ymarks/get_ymark_list.java, htroot/api/ymarks/get_ymark_list.xml, htroot/api/ymarks/get_ymark_tree.java, htroot/api/ymarks/get_ymark_tree.json, htroot/api/ymarks/get_ymark_tree.xml, htroot/api/ymarks/test.html, htroot/yacy/ui/css/jquery.treeview.css, source/de/anomic/data/YMarkIndex.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksHTMLImporter.java, source/de/anomic/data/YMarksXBELImporter.java
Wed Nov 03 07:47:02 CET 2010
by apfelmaennchen
more updates to the new bookmarks (ymarks)....
- split YMarkTables and YMarkIndex in two different classes
- HTML import is working properly
- XBEL import is still broken


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7292 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_html.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/YMarkIndex.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksHTMLImporter.java, source/de/anomic/data/YMarksXBELImporter.java
Wed Oct 27 15:21:18 CEST 2010
by orbiter
- better memory protection
- more logging
- little bit of refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7278 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/protocol/TimeoutRequest.java
Tue Oct 26 18:10:20 CEST 2010
by f1ori
* add a bit documentation to DigestURI, use DigestURI(string) instead of DigestURI(string, null)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7276 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/CacheResource_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigUpdate_p.java, htroot/Crawler_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImportOAIPMH_p.java, htroot/Load_RSS_p.java, htroot/QuickCrawlLink_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/WebStructurePicture_p.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/util/getpageinfo_p.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/blockrank/RCIEvaluation.java, source/de/anomic/yacy/graphics/OSMTile.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsPool.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/kelondro/data/meta/DigestURI.java
Tue Oct 26 00:16:07 CEST 2010
by orbiter
added a parent configuration option.
see /ConfigPortal.html
requested here:
http://forum.yacy-websuche.de/viewtopic.php?p=21099#p21099

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7271 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/index.java, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/search/SwitchboardConstants.java
Sun Oct 24 23:43:01 CEST 2010
by orbiter
- enhanced main release recognition
- yacybot user agent now includes the yacy network name (not the peer name!)
- refactoring and clean-up (mostly turned tab into spaces)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7266 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigUpdate_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/Network.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacyVersion.java, source/net/yacy/cora/document/MultiProtocolURI.java
Mon Oct 18 23:09:41 CEST 2010
by apfelmaennchen
added some more bookmark code...

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7261 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import.html, htroot/api/ymarks/import_html.java, htroot/api/ymarks/import_html.xml, source/de/anomic/data/WorkTables.java, source/de/anomic/data/YMarkTables.java, source/de/anomic/data/YMarksHTMLImporter.java
Mon Oct 18 13:35:09 CEST 2010
by orbiter
added a word counter statistic in condenser which is used by the did-you-mean to calculate best matches for given search words.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7258 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ViewFile.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/SnippetExtractor.java, source/net/yacy/document/parser/torrentParser.java
Mon Oct 18 10:09:59 CEST 2010
by orbiter
added user agent logging to access tracker

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7256 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.html, htroot/AccessTracker_p.java, htroot/AccessTracker_p.xml, htroot/IndexControlRWIs_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/QueryParams.java, source/net/yacy/cora/protocol/RequestHeader.java
Sat Oct 16 00:01:39 CEST 2010
by orbiter
- added a tag cloud to search results (using the topics)
- some refactoring of score classes
- added default package for new classes add_ymark and delete_ymark

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7251 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, htroot/Supporter.java, htroot/Surftips.java, htroot/yacy/ui/js/jquery.tagcloud.min.js, htroot/yacy/ui/js/jquery.tinysort.min.js, htroot/yacysearch.html, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/cora/storage/DynamicScore.java, source/net/yacy/cora/storage/IntScore.java, source/net/yacy/cora/storage/ScoreCluster.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/cora/storage/StaticScore.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/yacy.java
Thu Oct 14 13:40:02 CEST 2010
by orbiter
added more score cluster options and made score cluster usage more transparent

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7248 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Supporter.java, htroot/Surftips.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/cora/storage/DynamicScore.java, source/net/yacy/cora/storage/IntScore.java, source/net/yacy/cora/storage/ScoreCluster.java, source/net/yacy/cora/storage/ScoreMap.java, source/net/yacy/cora/storage/StaticScore.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/yacy.java
Wed Oct 13 00:02:10 CEST 2010
by orbiter
- enhanced DidYouMean computation using a faster count on index entries; this causes that results can be ranked better
- added limitations on DidYouMean result sets according to input and output string length

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7246 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/index.html, htroot/yacysearch.html, source/de/anomic/data/DidYouMean.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java
Tue Oct 12 11:45:15 CEST 2010
by orbiter
enhancements in did-you-mean guessing

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7243 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/index.html, htroot/suggest.java, htroot/yacysearch.html, htroot/yacysearch.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/DidYouMeanLibrary.java, source/net/yacy/document/geolocalization/GeonamesLocalization.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/document/geolocalization/OverarchingLocalization.java
Tue Oct 12 03:23:49 CEST 2010
by orbiter
Auto-Suggestions for YaCy Search:
- added a suggest servlet according to opensearch and firefox standard
- integrated the suggest servlet into opensearch description file
- integrated a autocomplete plugin for jquery
- added a autocomplete addition to the yacy search windows showing autosuggest queries

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7241 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/index.html, htroot/opensearchdescription.java, htroot/opensearchdescription.xml, htroot/suggest.java, htroot/suggest.json, htroot/yacy/search.java, htroot/yacy/ui/css/autocomplete.css, htroot/yacy/ui/js/jquery.autocomplete.js, htroot/yacysearch.html, source/de/anomic/search/ResultFetcher.java
Mon Oct 11 02:00:10 CEST 2010
by orbiter
custom + generic skins:
- added a generic skin which is filled with actual color assignment using a servlet
- enabled css servlets
- added a generic color scheme in configuration file
- added configuration input in Customization/Appearance servlet
- added a jquery color picker widget
- placed color picked widget to input field of generic colour definition input fields

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7235 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigAppearance_p.html, htroot/ConfigAppearance_p.java, htroot/env/style.java, htroot/env/templates/header.template, htroot/js/yacyinteractive.js, htroot/yacy/ui/colorpicker_images/Thumbs.db, htroot/yacy/ui/colorpicker_images/blank.gif, htroot/yacy/ui/colorpicker_images/colorpicker_background.png, htroot/yacy/ui/colorpicker_images/colorpicker_hex.png, htroot/yacy/ui/colorpicker_images/colorpicker_hsb_b.png, htroot/yacy/ui/colorpicker_images/colorpicker_hsb_h.png, htroot/yacy/ui/colorpicker_images/colorpicker_hsb_s.png, htroot/yacy/ui/colorpicker_images/colorpicker_indic.gif, htroot/yacy/ui/colorpicker_images/colorpicker_overlay.png, htroot/yacy/ui/colorpicker_images/colorpicker_rgb_b.png, htroot/yacy/ui/colorpicker_images/colorpicker_rgb_g.png, htroot/yacy/ui/colorpicker_images/colorpicker_rgb_r.png, htroot/yacy/ui/colorpicker_images/colorpicker_select.gif, htroot/yacy/ui/colorpicker_images/colorpicker_submit.png, htroot/yacy/ui/colorpicker_images/custom_background.png, htroot/yacy/ui/colorpicker_images/custom_hex.png, htroot/yacy/ui/colorpicker_images/custom_hsb_b.png, htroot/yacy/ui/colorpicker_images/custom_hsb_h.png, htroot/yacy/ui/colorpicker_images/custom_hsb_s.png, htroot/yacy/ui/colorpicker_images/custom_indic.gif, htroot/yacy/ui/colorpicker_images/custom_rgb_b.png, htroot/yacy/ui/colorpicker_images/custom_rgb_g.png, htroot/yacy/ui/colorpicker_images/custom_rgb_r.png, htroot/yacy/ui/colorpicker_images/custom_submit.png, htroot/yacy/ui/colorpicker_images/select.png, htroot/yacy/ui/colorpicker_images/select2.png, htroot/yacy/ui/colorpicker_images/slider.png, htroot/yacy/ui/css/colorpicker.css, htroot/yacy/ui/js/jquery.colorpicker.js, htroot/yacysearch.java, skins/generic_pd.css, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/http/server/HTTPDFileHandler.java
Sat Oct 09 10:55:57 CEST 2010
by orbiter
more performance hacks
this makes YaCy search results VERY fast for all verify=false search cases
and it enhances the search speed also for all other snippet-fetch cases.
With this change my peer performed 100 Queries Per Second (!!!) while doing 10 queries simultanously (!!!)
in an intranet index of 20000 URLs on my 16-core Mac

Check this yourself by doing:
cd bin
./searchtestmulti.sh
after finishing the run, divide 1000 by the given time per query (which is the qps for one thread)
and then multiply again by 10 (because 10 search threads has been started)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7231 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/LargeNumberCache.java, source/net/yacy/document/Phrase.java, source/net/yacy/document/SnippetExtractor.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/order/Digest.java
Sat Oct 09 01:50:28 CEST 2010
by orbiter
performance hacks for better search performance

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7230 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/search/QueryParams.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/storage/ComparableARC.java, source/net/yacy/cora/storage/HashARC.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/document/parser/html/AbstractScraper.java, source/net/yacy/document/parser/html/CharacterCoding.java, source/net/yacy/document/parser/html/ContentScraper.java
Fri Oct 08 12:54:13 CEST 2010
by orbiter
- enhanced intranet search speed
- enhanced intranet portscan speed (better time-out)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7227 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearchitem.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/search/RankingProcess.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/kelondro/data/meta/DigestURI.java
Wed Oct 06 02:00:23 CEST 2010
by mikeworks
de.lng: Added some more untranslated strings I found and uncommented old ones that were removed
terminal_p.html: Put back the old ID which was really easy to find
IndexCreate.js: Because XHTML 1.0 Strict does not allow name tags for some elements rewrote most element access functions to use getElementById
Table_API_p.html and all other html pages: Some XHTMl 1.0 Strict fixes, changed checkAll javascript, marked the first row with checkboxes as unsortable where applicable
Table_API_p.java and all other java pages: URLencoded lines with possible ampersands & -> & for validation XHTML 1.0 Strict sourcecode
--> All Index Create pages should validate now. Hope I did not break anything else (too much :-)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7225 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.html, htroot/ContentIntegrationPHPBB3_p.html, htroot/CrawlStartExpert_p.html, htroot/CrawlStartSite_p.html, htroot/IndexImportOAIPMHList_p.html, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.html, htroot/IndexImportWikimedia_p.html, htroot/Load_MediawikiWiki.html, htroot/Load_PHPBB3.html, htroot/Load_RSS_p.html, htroot/Load_RSS_p.java, htroot/Table_API_p.html, htroot/Table_API_p.java, htroot/js/IndexCreate.js, htroot/terminal_p.html, locales/de.lng
Mon Oct 04 13:54:48 CEST 2010
by orbiter
- enhancements for search speed
- bug fixes in many classes including basic data structure classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7217 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/env/templates/header.template, htroot/yacy/search.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/cora/storage/WeakPriorityBlockingQueue.java, source/net/yacy/document/Parser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/repository/LoaderDispatcher.java
Fri Oct 01 01:57:58 CEST 2010
by orbiter
fixed a number of small bugs:
- better crawl star for files paths and smb paths
- added time-out wrapper for dns resolving and reverse resolving to prevent blockings
- fixed intranet scanner result list check boxes
- prevented htcache usage in case of file and smb crawling (not necessary, documents are locally available)
- fixed rss feed loader
- fixes sitemap loader which had not been restricted to single files (crawl-depth must be zero)
- clearing of crawl result lists when a network switch was done
- higher maximum file size for crawler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7214 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStartIntranet_p.html, htroot/CrawlStartSite_p.html, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, htroot/SettingsAck_p.java, htroot/api/util/getpageinfo_p.java, htroot/js/IndexCreate.js, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/storage/ARC.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/kelondro/table/Table.java
Thu Sep 30 14:50:34 CEST 2010
by orbiter
replaced auto-dom filter with easy-to-understand Site Link-List crawler option
- nobody understand the auto-dom filter without a lenghtly introduction about the function of a crawler
- nobody ever used the auto-dom filter other than with a crawl depth of 1
- the auto-dom filter was buggy since the filter did not survive a restart and then a search index contained waste
- the function of the auto-dom filter was in fact to just load a link list from the given start url and then start separate crawls for all these urls restricted by their domain
- the new Site Link-List option shows the target urls in real-time during input of the start url (like the robots check) and gives a transparent feed-back what it does before it can be used
- the new option also fits into the easy site-crawl start menu

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7213 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlProfileEditor_p.html, htroot/CrawlProfileEditor_p.java, htroot/CrawlProfileEditor_p.xml, htroot/CrawlStartExpert_p.html, htroot/CrawlStartSite_p.html, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, htroot/api/util/getpageinfo_p.java, htroot/api/util/getpageinfo_p.xml, htroot/js/IndexCreate.js, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java
Thu Sep 30 01:11:29 CEST 2010
by mikeworks
Collage.html and Collage.java: Added <p> in body before images for XHTML 1.0 Transitional validationg and alt tag to images as well as closing tag <img (...) />
terminal_p.html: Set new link for starting a crawl to CrawlStartSite_p.html and replaced the old embed object of the Among.us Flash object by their new JS which takes care of adding the object correctly
de.lng: Moved the translations for the JS part from yacyinteractive.html to the yacyinteractive.js part
--> Terminal page is now valid XHTML 1.0 Transitional


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7211 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Collage.html, htroot/Collage.java, htroot/terminal_p.html, locales/de.lng
Wed Sep 29 22:06:39 CEST 2010
by mikeworks
ConfigLiveSearch.html: Fixed some HTML problems to validate at least XHTML 1.0 Transitional - strict is not possible because iframes are used. Replacing iframes with embedded object tag does not work in IE
ConfigPortal.html: Fixed some HTML problems to validate at least XHTML 1.0 Transitional - for strict the target attribute of the a link has to be removed
yacyinteractive.html: Moved all JS code to an external yacyinteractive.js file in JS folder
yacysearch.html: Removed embedded scripts from in between the body tags - now everything is loaded in the header
de.lng: Just in case JS files will be parsed at some point added translation for yacyinteractive.html result counter

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7208 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigLiveSearch.html, htroot/ConfigPortal.html, htroot/js/yacyinteractive.js, htroot/yacyinteractive.html, htroot/yacysearch.html, locales/de.lng
Mon Sep 27 16:54:32 CEST 2010
by orbiter
- moved yacybot user agent string definition to MultiProtocolURI since there are basic access mechanisms where the bot string is needed
- migrated the 'yacy' user agent to 'yacybot' in many client methods since the 'yacy' user agent is only used for the proxy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7199 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/Network.java, htroot/sharedBlacklist_p.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/Switchboard.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/Search.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/yacy.java
Wed Sep 22 22:50:02 CEST 2010
by orbiter
simplified snippet computation process and separated the algorithm into two classes
also enhances selection criteria for best snippet line computation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7182 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ViewFile.java, htroot/yacysearch.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/storage/WeakPriorityBlockingQueue.java, source/net/yacy/document/Document.java, source/net/yacy/document/SnippetExtractor.java
Tue Sep 21 23:48:42 CEST 2010
by orbiter
fixed several search bugs

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7180 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ViewImage.java, htroot/yacysearch.html, htroot/yacysearch.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/YaCySearchClient.java, source/net/yacy/document/parser/images/genericImageParser.java
Mon Sep 20 15:17:25 CEST 2010
by orbiter
enhanced search speed:
- better control of number of running search threads
- no time-out waiting time when no ranking feeding takes place
- local search queries by a remote peer may be faster up to 300 milliseconds
- a local search may even be faster

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7176 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segment.java, source/net/yacy/cora/storage/WeakPriorityBlockingQueue.java, source/net/yacy/kelondro/table/SplitTable.java
Mon Sep 20 10:36:48 CEST 2010
by orbiter
terminated migration from apache httpclient-3.1 to 4.1:
- remove the library
- added two classes from the httpclient-3.1 library as source code to YaCy because these classes were used by the YaCy HTTP Server
- modified the added classes ChunkedInputStream and ContentLengthInputStream in such a way that:
 * there are no more dependencies to httpclient-3.1
 * these classes had been simplified to serve only the purpose for the YaCy httpd

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7171 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, source/de/anomic/http/server/ChunkedInputStream.java, source/de/anomic/http/server/ContentLengthInputStream.java, source/de/anomic/http/server/HTTPDemon.java
Sat Sep 18 02:18:52 CEST 2010
by low012
*) Changed image in Steering.html from linked image to embedded image because shutdown is so fast now, browsers can't load image before Yacy instance is gone already. Had to make image smaller since IE does not accept large Base64 encoded images.
*) Decreases wait time in Steering.html before first check since 
*) HTML fixes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7165 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigProperties_p.java, htroot/Status.html, htroot/Steering.html, htroot/yacy/ui/index.html, htroot/yacyinteractive.html, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchtrailer.java
Fri Sep 17 12:20:04 CEST 2010
by orbiter
- fixes for concurrency exceptions and failed database integrity verification
- added link to yacystats peer when peer is more than one day old

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7164 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/AccessTracker_p.java, htroot/CrawlResults.java, htroot/env/templates/header.template, htroot/proxymsg/error.html, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/server/serverCore.java, source/net/yacy/kelondro/blob/HeapReader.java
Fri Sep 17 01:00:07 CEST 2010
by orbiter
- some layout and text enhancement for site crawl start
- Quix0rs patch from http://forum.yacy-websuche.de/viewtopic.php?p=20839#p20839 (parts)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7163 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/CrawlStart_p.html, htroot/env/base.css, htroot/env/templates/submenuIndexCreate.template, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/server/serverAccessTracker.java, source/de/anomic/server/serverSwitch.java
Thu Sep 16 11:34:17 CEST 2010
by orbiter
enhanced remote crawling:
- 300 ppm is default now (but this is switched off by default; if you switch it on you may want more traffic?)
- better timing for busy queue
- better amount of remote url retrieval
- better time-out values
- better tracking of availability of remote crawl urls
- more logging for result of receipt sending

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7159 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, defaults/yacy.network.freeworld.unit, htroot/PerformanceQueues_p.java, htroot/RemoteCrawl_p.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/kelondro/table/Table.java
Thu Sep 16 02:39:05 CEST 2010
by orbiter
- removed the remote crawl option from the network configuration submenu and
- added a remote crawl menu item to the index create menu. This menu also shows a list of peers that provide remote crawl urls
- set remote crawl option by default to off. This option may be important but it also confuses first-time users


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7158 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigNetwork_p.html, htroot/ConfigNetwork_p.java, htroot/RemoteCrawl_p.html, htroot/RemoteCrawl_p.java, htroot/env/templates/submenuIndexCreate.template, source/de/anomic/yacy/yacySeedDB.java
Wed Sep 15 12:43:14 CEST 2010
by orbiter
- corrected return value of put() methods (not used anywhere, so it did not harm before)
- added use of LookAheadIterator which should prevent mistakes when coding iterators with embedded iterators
- added a fail-safe reaction in case of database corruption using iterators over database elements (no interruption then)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7154 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/Index.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java
Tue Sep 14 23:03:50 CEST 2010
by orbiter
- code cleanup / added debug line for further investigation in HTTPDemon.parseMultipart
- changed data structure for sorting in search which performs better in that specific case (too many updates)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7150 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/protocol/Domains.java
Tue Sep 14 16:32:24 CEST 2010
by orbiter
performance hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7147 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/WebStructurePicture_p.java, htroot/api/webstructure.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/blockrank/RCIEvaluation.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/Index.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/ScoreCluster.java
Mon Sep 13 11:33:04 CEST 2010
by orbiter
enhanced search event logging (to be used for further improvements)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7140 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceSearch_p.html, htroot/PerformanceSearch_p.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/graphics/ProfilingGraph.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/util/EventTracker.java, source/net/yacy/kelondro/util/MemoryTracker.java
Thu Sep 09 17:30:25 CEST 2010
by orbiter
redesign of the SortStack and SortStore classes:
created a WeakPriorityBlockingQueue as special implementation
of a PriorityBlockingQueue with a weak object binding.
- better abstraction of ordering technique
- fixed some bugs according to result numbering (distinguish different counters in Queue)
- fixed a ordering bug in post-ranking (ordering was decreased instead of increased)
- reversed ordering numbering using a reversed ordering. The higher the ranking number the better (now).

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7128 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/search.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/net/yacy/cora/storage/WeakPriorityBlockingQueue.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java
Tue Sep 07 19:13:47 CEST 2010
by orbiter
- replaced pdfbox and fontbox version 1.1.0 with 1.2.1
- added some clear statements that shall clear static cache size within the pdfbox library
- the pdfbox library contains a memory leak; it is unsafe to run a peer with pdf parser permanently on.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7120 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, addon/yacyInit.m4, build.xml, htroot/Threaddump_p.java, lib/fontbox-1.2.1.jar, lib/pdfbox-1.2.1.jar, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/http/client/Cache.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/logging/ThreadDumpGenerator.java, source/net/yacy/kelondro/util/ByteArray.java, startYACY.bat, startYACY.sh
Thu Sep 02 21:24:22 CEST 2010
by orbiter
change in handling of the all-visible home path for storage in YaCy:
the home path can now be distinguished between
- data home; the path where the DATA directory is created
- application home; everything else
This will make it possible to store application data on Mac releases within the
~/Library/YaCy
directory; a place where Mac applications write their data.
Similar techniques will be possible for debian and windows.
To use the new data path, YaCy can be started with
-start <data path>
or
-gui <data path>


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7092 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: addon/YaCy.app/Contents/Info.plist, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/BlogComments.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigUpdate_p.java, htroot/IndexControlURLs_p.java, htroot/Messages_p.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/Steering.java, htroot/Threaddump_p.java, htroot/ViewImage.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/cytag.java, htroot/mediawiki_p.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/yacy/list.java, htroot/yacy/message.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/data/translator.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/yacyAccessible.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/kelondro/logging/Log.java, source/net/yacy/migration.java, source/net/yacy/yacy.java
Tue Aug 31 17:47:47 CEST 2010
by orbiter
redesign of crawl profiles data structure. target will be:
- permanent storage of auto-dom statistics in profile
- storage of profiles in WorkTable data structure
not finished yet. No functional change yet.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7088 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlProfileEditor_p.java, htroot/Crawler_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/WatchWebStructure_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/WorkTables.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/repository/LoaderDispatcher.java
Thu Aug 26 18:01:45 CEST 2010
by orbiter
added two more tables in rss reader interface:
- fresh recorded rss feeds (not yet loaded or in scheduler)
- rss feeds in scheduler
The first list has a button that can be used to place rss feeds into the scheduler
The second list has a button to delete rss feeds from the scheduler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7074 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Load_RSS_p.html, htroot/Load_RSS_p.java, source/de/anomic/data/WorkTables.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Tables.java
Wed Aug 25 20:24:54 CEST 2010
by orbiter
Support for indexing of RSS feeds!
- added a scanning in html parser for rss feeds
- storage of rss feed addresses, can be viewed with http://localhost:8080/Tables_p.html?table=rss
- rss items retrieved by http://localhost:8080/Load_RSS_p.html (in Index Creation menu) can be selected and indexed
- a rss feed retrieved in http://localhost:8080/Load_RSS_p.html can now be fully indexed
- indexing of rss feeds can be placed in scheduler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7073 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Load_RSS_p.html, htroot/Load_RSS_p.java, htroot/Tables_p.html, htroot/Tables_p.java, htroot/api/table_p.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/kelondro/blob/Tables.java
Tue Aug 24 14:36:56 CEST 2010
by orbiter
- MapHeap now implements Map<byte[], Map<String, String>>
- refactoring of method names to comply with Map method names

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7072 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/data/BookmarkDate.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/http/client/Cache.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Stack.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java
Tue Aug 24 01:38:03 CEST 2010
by orbiter
implemented the Map interface in the ARC classes so it will be possible to instantiate ARCs as
Map<byte[], Map<String, byte[]>>
Because such Maps with byte[] keys cannot be stored in hash maps (bad hashing on byte[])
another ARC with comparable Maps has been added

This will make it possible to move the HTCache class 'Cache' into the cora package because that
class may be used either with RAM caches (ARCs) or with file-based caches (BEncodedHeaps)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7071 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/cora/storage/ARC.java, source/net/yacy/cora/storage/ComparableARC.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/HashARC.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/kelondro/blob/BEncodedHeap.java
Mon Aug 23 14:32:02 CEST 2010
by orbiter
preparations to move the HTCache into cora:
- move the header framework classes to cora
- move the ARC caching classes to cora
- refactoring of code to call these classes from cora

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7068 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: bin/graphicstest.sh, htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigHeuristics_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigLiveSearch.java, htroot/ConfigNetwork_p.java, htroot/ConfigParser.java, htroot/ConfigPortal.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.java, htroot/Help.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateDomainCrawl_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.java, htroot/IndexImportWikimedia_p.java, htroot/IndexShare_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/Load_RSS_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/Performance_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Table_API_p.java, htroot/Table_RobotsTxt_p.java, htroot/Tables_p.java, htroot/Threaddump_p.java, htroot/Trails.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/tags/editTag_p.java, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/config_p.java, htroot/api/feed.java, htroot/api/latency_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/table_p.java, htroot/api/timeline.java, htroot/api/trail_p.java, htroot/api/util/getpageinfo_p.java, htroot/api/util/ynetSearch.java, htroot/api/version.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/autoconfig.java, htroot/compare_yacy.java, htroot/cytag.java, htroot/imagetest.java, htroot/index.java, htroot/mediawiki_p.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, maptest, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/userDB.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/net/UPnP.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/server/servletProperties.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/RequestHeader.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/storage/ARC.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/util/DateFormatter.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/yacy.java, test/de/anomic/kelondro/util/DateFormatterTest.java
Mon Aug 23 13:41:12 CEST 2010
by orbiter
- removed dependencies in header framework:
  moved http date methods from DateFormatter to HeaderFramework
  changed logging to log4j
- added ftp load access to MultiProtocolURI
- ensured termination of RSS feed iteration

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7067 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Load_RSS_p.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HeaderFramework.java, source/de/anomic/http/server/RequestHeader.java, source/de/anomic/http/server/ResponseHeader.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/services/Search.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/kelondro/util/DateFormatter.java, test/de/anomic/kelondro/util/DateFormatterTest.java
Mon Aug 23 03:08:56 CEST 2010
by orbiter
refactoring towards a unified loading mechanism for MultiProtocolURIs

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7065 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/SettingsAck_p.java, htroot/sharedBlacklist_p.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/RequestHeader.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/HTTPConnector.java, source/net/yacy/cora/protocol/http/HttpConnector.java, source/net/yacy/cora/services/Search.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/kelondro/util/Domains.java
Mon Aug 23 00:32:39 CEST 2010
by orbiter
refactoring of the cora package to get subpackages for http and ftp (smb to come)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7063 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/WorkTables.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Switchboard.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFtp.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/protocol/ConnectionInfo.java, source/net/yacy/cora/protocol/HttpConnector.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/protocol/http/GzipDecompressingEntity.java, source/net/yacy/cora/protocol/http/GzipRequestInterceptor.java, source/net/yacy/cora/protocol/http/GzipResponseInterceptor.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/ProxySettings.java, source/net/yacy/yacy.java
Sun Aug 22 19:38:27 CEST 2010
by sixcooler
... migrating to HttpComponents-Client-4.x ...
The Client is dead, long live the Client!
(no references to the old client)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7060 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/http/client/Client.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/protocol/Client.java, source/net/yacy/cora/protocol/ConnectionInfo.java, source/net/yacy/yacy.java
Fri Aug 20 13:30:02 CEST 2010
by orbiter
- added a new RSS reader interface. This is not finished but you can now load and look at RSS feeds. It will be used to index RSS feeds in a way that is appropriate for such kind of data.
- refactoring of Mediawiki and PHPBB3 loader interface names (just renamed)
- removed two old not used RSS loader interfaces
- fixed a bug in RSS parser library of cora
- added a new RSS parser component to the set of yacy document parsers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7053 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Load_MediawikiWiki.html, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.html, htroot/Load_PHPBB3.java, htroot/Load_RSS_p.html, htroot/Load_RSS_p.java, htroot/env/templates/submenuIndexCreate.template, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/rssParser.java
Fri Aug 20 01:52:38 CEST 2010
by orbiter
added the new crawl scheduling function to the crawl start menu:
- the scheduler extends the option for re-crawl timing. Many people misunderstood the re-crawl timing feature because that was just a criteria for the url double-check and not a scheduler. Now the scheduler setting is combined with the re-crawl setting and people will have the choice between no re-crawl, re-crawl as was possible so far and a scheduled re-crawl. The 'classic' re-crawl time is set automatically when the scheduling function is selected
- removed the bookmark-based scheduler. This scheduler was not able to transport all attributes of a crawl start and did therefore not support special crawling starts i.e. for forums and wikis
- since the old scheduler was not aber to crawl special forums and wikis, the must-not-match filter was statically fixed to all bad pages for these special use cases. Since the new scheduler can handle these filters, it is possible to remove the default settings for the filters
- removed the busy thread that was used to trigger the bookmark-based scheduler
- removed the crontab for the bookmark-based scheduler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7051 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/CrawlStart_p.html, htroot/Crawler_p.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/Table_API_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/blob/Tables.java
Thu Aug 19 14:13:54 CEST 2010
by orbiter
added a scheduler based on API actions
- every process that is monitored with the API Steering interface can now be scheduled!
- added input methods in Steering interface to set a scheduling time
- added a view on the steering api that shows only crawl jobs inside the Crawl Profile servlet
- added a scheduling call process in the cleanup process handler that triggers the scheduled processes
This causes that the cleanup now also looks for scheduled processes. Such processes are therefore not executed at
the same time as given in the target execution time but they will be executed within the cleanup process time window.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7050 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlProfileEditor_p.html, htroot/CrawlProfileEditor_p.java, htroot/Table_API_p.html, htroot/Table_API_p.java, htroot/Tables_p.html, htroot/env/templates/submenuCrawlMonitor.template, source/de/anomic/data/WorkTables.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/util/BEncoder.java, source/net/yacy/kelondro/util/DateFormatter.java
Wed Aug 18 17:56:38 CEST 2010
by orbiter
- modified api-call interface to record api calls with references to api-call database (carries pk)
- added recording date, last execution date and next execution date for a scheduler (scheduler to be implemented next)
- extended database access methods for more data formats, especially for date insert/retrieval
- extended 'Steering' interface to show new database fields
- migrated Steering to new http client
- extended cora http client to transmit authentication and also added some convenience methods (http response code)
- simplified database back-end (not so much specialized methods for multiple properties)
- extended date formatter to produce a special format to show dates in html (  in spaces of date format)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7049 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.xml, htroot/Table_API_p.html, htroot/Table_API_p.java, htroot/Tables_p.java, htroot/api/table_p.java, source/de/anomic/data/WorkTables.java, source/net/yacy/cora/protocol/Client.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/BEncoder.java, source/net/yacy/kelondro/util/DateFormatter.java
Tue Aug 17 12:44:00 CEST 2010
by orbiter
- the web visualization can now be generated in custom color
- added input fields in WatchWebStructure_p.html
- introduced enum classes for Draw Mode and Filter Mode

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7044 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/SearchEventPicture.java, htroot/WatchWebStructure_p.html, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/imagetest.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/graphics/OSMTile.java, source/net/yacy/visualization/AnimGifEncoder.java, source/net/yacy/visualization/Captcha.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/DemoApplet.java, source/net/yacy/visualization/GraphPlotter.java, source/net/yacy/visualization/RasterPlotter.java
Wed Aug 11 11:54:18 CEST 2010
by orbiter
- better url double check in crawler
- more logging for error urls

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7032 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/table/Table.java
Tue Aug 10 01:41:17 CEST 2010
by orbiter
- removed unused network oanet
- added new network definition 'allip' which can be used in networks where intranet and internet-addresses shall be indexed
- added a auto-switch-off for global search if there are no global peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7030 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.network.allip.unit, defaults/yacy.network.webportal.unit, defaults/yacy.networks, htroot/yacysearch.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/search/SearchEvent.java, source/net/yacy/yacy.java
Thu Aug 05 13:11:15 CEST 2010
by orbiter
added a Macintosh OSX application wrapper.
You can now create a YaCy Mac application like this:
- build a new version of YaCy
- take everything that is in the yacy home directory (from the fresh build) and move it into a copy of the YaCy.app directory under YaCy.app/Contents/Resources/Java/
- ready: double-click the YaCy.app icon and the new YaCy gui starts, then the browsers is also opened.
- if the Mac application is terminated, also the YaCy demon should be stopped automatically and vice versa. This is currently not really good working and needs some more care.
A build script for this is missing. Does anyone know how to do that in ant?

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7023 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: addon/YaCy.app/Contents/Info.plist, addon/YaCy.app/Contents/MacOS/JavaApplicationStub, addon/YaCy.app/Contents/PkgInfo, addon/YaCy.app/Contents/Resources/YaCyLogo2008.icns
Thu Aug 05 12:43:03 CEST 2010
by orbiter
added a very early test version of a YaCy gui component.
The gui currently does nothing else than providing a search window that sends the search string to the browser
The gui is started when YaCy is started with the option -g or --gui, like
./startYACY.sh -g
The gui will primary be used to provide a 'real' macintosh version that can be started and operated like any other macintosh application. A special mac application wrapper will follow.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7021 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/search/Switchboard.java, source/net/yacy/gui/InfoPage.java, source/net/yacy/gui/YaCyApp.java, source/net/yacy/gui/framework/Application.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/gui/framework/Layout.java, source/net/yacy/gui/framework/Operation.java, source/net/yacy/gui/framework/Switchboard.java, source/net/yacy/yacy.java, startYACY.command, startYACY.sh
Wed Aug 04 15:33:12 CEST 2010
by orbiter
better memory management and slightly less (in total and temporary) RAM allocation:
- confirm that database objects that are not supposed to grow do not have a index memory management that is designed for growth
- changed index sorting method in such a way that it allocates less objects during quicksort
- database classes classes renaming (shorter, naming addresses that objects hold in RAM)
- added a large number of asserts to check if objects actually take the RAM that they should have


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7019 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceMemory_p.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/http/client/Cache.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/yacy/yacyNewsDB.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/Index.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java
Tue Aug 03 06:58:48 CEST 2010
by orbiter
- enhanced concurrency in database index access for multicore
- added statistics about database index caches in PerformanceMemory_p.html
- adoped many classes to use the new statistics
- added missing close statements

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7018 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/io/CharBuffer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java
Tue Jul 27 08:48:01 CEST 2010
by orbiter
added generics to UPnP classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7002 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/upnp/Discovery.java, source/net/yacy/upnp/DiscoveryAdvertisement.java, source/net/yacy/upnp/DiscoveryListener.java, source/net/yacy/upnp/ServiceEventMessageParser.java, source/net/yacy/upnp/ServicesEventing.java, source/net/yacy/upnp/devices/UPNPDevice.java, source/net/yacy/upnp/impls/InternetGatewayDevice.java, source/net/yacy/upnp/messages/ActionMessage.java, source/net/yacy/upnp/messages/ActionResponse.java, source/net/yacy/upnp/services/ServiceAction.java, source/net/yacy/upnp/services/ServiceStateVariable.java, source/net/yacy/upnp/services/UPNPService.java
Tue Jul 27 03:16:26 CEST 2010
by sixcooler
... migrating to HttpComponents-Client-4.x ...
(gzip decompression, httploader, robots, ...)

+ enable proxy-crawling while log is fine

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7001 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/Network.java, htroot/api/queues_p.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/search/Switchboard.java, source/net/yacy/cora/protocol/Client.java, source/net/yacy/cora/protocol/GzipDecompressingEntity.java, source/net/yacy/cora/protocol/GzipRequestInterceptor.java, source/net/yacy/cora/protocol/GzipResponseInterceptor.java
Sun Jul 25 15:19:16 CEST 2010
by mikeworks
ConfigUpdate_p.html and ConfigUpdate_p.java: Added check for downloaded releases and disabled buttons in case no new releases available
de.lng: Updated German translation for additional String in ConfigUpdate_p.html
XHTML 1.0 Strict fixes for all the other .html files
yacy/ui/css/yacyui-portalsearch.css: added .hidden class that was removed from ConfigProperties_p.html
Switchboard.java: Added URL for thread Remote Crawl Job and set URL for Remote Crawl URL Loader to null to fix empty href=""

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6996 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAccounts_p.html, htroot/ConfigNetwork_p.html, htroot/ConfigProperties_p.html, htroot/ConfigUpdate_p.html, htroot/ConfigUpdate_p.java, htroot/PerformanceMemory_p.html, htroot/Tables_p.html, htroot/env/base.css, htroot/yacy/ui/css/yacyui-portalsearch.css, locales/de.lng, source/de/anomic/search/Switchboard.java
Sat Jul 24 17:26:53 CEST 2010
by mikeworks
ConfigBasic.html: XHTML 1.0 Strict fixes
DictionaryLoader_p.html: Filled <dt> elements to eliminate warnings
Moved CSS for portalsearch field from header to metas template because it belongs in the <head>er
yacui-portalsearch.css Added #yacylivesearch form { display: inline; } because HTML 1.0 Strict does not allow <form><input> and the added <p> would otherwise provoke a line break
de.lng: Updates translations for added <dt> elements and deactivated statement in DictionaryLoader_p.html


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6994 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigBasic.html, htroot/DictionaryLoader_p.html, htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/yacy/ui/css/yacyui-portalsearch.css, locales/de.lng
Sat Jul 24 15:48:45 CEST 2010
by mikeworks
Added Generics to new net.yacy.upnp.* classes to eliminate compiler warnings
Added @Deprecated for deprecated functions getIPDevices and getPPPDevices in class InternetGatewayDevice
Changed debug statement in Domains.java and corrected filename in comments header

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6993 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/kelondro/util/Domains.java, source/net/yacy/upnp/Discovery.java, source/net/yacy/upnp/DiscoveryAdvertisement.java, source/net/yacy/upnp/DiscoveryListener.java, source/net/yacy/upnp/HttpResponse.java, source/net/yacy/upnp/ServiceEventMessageParser.java, source/net/yacy/upnp/ServicesEventing.java, source/net/yacy/upnp/devices/UPNPDevice.java, source/net/yacy/upnp/devices/UPNPRootDevice.java, source/net/yacy/upnp/impls/InternetGatewayDevice.java, source/net/yacy/upnp/messages/ActionMessage.java, source/net/yacy/upnp/messages/ActionResponse.java, source/net/yacy/upnp/services/ServiceAction.java, source/net/yacy/upnp/services/ServiceStateVariable.java, source/net/yacy/upnp/services/UPNPService.java
Fri Jul 23 01:08:37 CEST 2010
by sixcooler
... migrating to HttpComponents-Client-4.x ...

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6989 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/sharedBlacklist_p.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Switchboard.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/cora/protocol/Client.java, source/net/yacy/cora/protocol/HttpConnector.java
Thu Jul 22 20:02:16 CEST 2010
by lotus
including sbbi upnplib as source again
http://www.sbbi.net/site/upnp/index.html

renamed package to yacy
all options are also named "yacy" instead of "sbbi"

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6986 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, source/de/anomic/net/UPnP.java, source/net/yacy/upnp/Discovery.java, source/net/yacy/upnp/DiscoveryAdvertisement.java, source/net/yacy/upnp/DiscoveryEventHandler.java, source/net/yacy/upnp/DiscoveryListener.java, source/net/yacy/upnp/DiscoveryResultsHandler.java, source/net/yacy/upnp/HttpResponse.java, source/net/yacy/upnp/JXPathParser.java, source/net/yacy/upnp/ServiceEventHandler.java, source/net/yacy/upnp/ServiceEventMessageParser.java, source/net/yacy/upnp/ServiceEventSubscription.java, source/net/yacy/upnp/ServicesEventing.java, source/net/yacy/upnp/devices/DeviceIcon.java, source/net/yacy/upnp/devices/UPNPDevice.java, source/net/yacy/upnp/devices/UPNPRootDevice.java, source/net/yacy/upnp/impls/InternetGatewayDevice.java, source/net/yacy/upnp/messages/ActionMessage.java, source/net/yacy/upnp/messages/ActionMessageResponseParser.java, source/net/yacy/upnp/messages/ActionResponse.java, source/net/yacy/upnp/messages/StateVariableMessage.java, source/net/yacy/upnp/messages/StateVariableResponse.java, source/net/yacy/upnp/messages/StateVariableResponseParser.java, source/net/yacy/upnp/messages/UPNPMessageFactory.java, source/net/yacy/upnp/messages/UPNPResponseException.java, source/net/yacy/upnp/services/ISO8601Date.java, source/net/yacy/upnp/services/ServiceAction.java, source/net/yacy/upnp/services/ServiceActionArgument.java, source/net/yacy/upnp/services/ServiceStateVariable.java, source/net/yacy/upnp/services/ServiceStateVariableTypes.java, source/net/yacy/upnp/services/UPNPService.java, source/net/yacy/upnp/version.properties
Sun Jul 18 22:14:20 CEST 2010
by orbiter
better (and corrected) recognition of intranet and internet-addresses. This corrects the isLocal property that is used by network definitions to restrict index ranges to local and global addresses. Address locations (intranet or internet) had been partly identified by the top level domain of the host address. Since intranet addresses can also be addressed using a host name that is in a country domain it is necessary to do a dns resolving for each check. The check is supported by a local dns cache so the intranet/internet check should not affect network traffic too much. To ensure that the cache works properly the cache class was upgraded to better concurrency data structures.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6977 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/Latency.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/kelondro/index/ARC.java, source/net/yacy/kelondro/index/ConcurrentARC.java, source/net/yacy/kelondro/index/SimpleARC.java, source/net/yacy/kelondro/util/Domains.java, source/net/yacy/repository/LoaderDispatcher.java
Tue Jul 13 01:07:05 CEST 2010
by sixcooler
start migrating to HttpComponents-Client-4.x
see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2872

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6965 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.xml, lib/apache-mime4j-0.6.License, lib/apache-mime4j-0.6.jar, lib/httpclient-4.0.1.License, lib/httpclient-4.0.1.jar, lib/httpcore-4.0.1.License, lib/httpcore-4.0.1.jar, lib/httpmime-4.0.1.License, lib/httpmime-4.0.1.jar, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNetwork.java, source/net/yacy/cora/protocol/Client.java, source/net/yacy/cora/protocol/HttpConnector.java
Tue Jun 29 21:20:45 CEST 2010
by orbiter
redesign of parser interface:
some file types are containers for several files. These containers had been parsed in such a way that the set of resulting parsed content was merged into one single document before parsing. Using this parser infrastructure it is not possible to parse document containers that contain individual files. An example is a rss file where the rss messages can be treated as individual documents with their own url reference. Another example is a surrogate file which was treated with a special operation outside of the parser infrastructure.
This commit introduces a redesigned parser interface and a new abstract parser implementation. The new parser interface has now only one entry point and returns always a set of parsed documents. In case of single documents the parser method returns a set of one documents.
To be compliant with the new interface, the zip and tar parser had been also completely redesigned. All parsers are now much more simple and cleaner in its structure. The switchboard operations had been extended to operate with sets of parsed files, not single parsed files.
additionally, parsing of jar manifest files had been added.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6955 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/httpd.mime, htroot/Bookmarks.java, htroot/ConfigParser.java, htroot/RSSLoader_p.java, htroot/ViewFile.java, htroot/api/feed.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyChannel.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Classification.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/ImageParser.java, source/net/yacy/document/Parser.java, source/net/yacy/document/Phrase.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/repository/LoaderDispatcher.java, test/de/anomic/document/ParserTest.java
Mon Jun 28 18:25:14 CEST 2010
by orbiter
migrated all my LGPL 3 -licensed files to the LGPL 2.1 because LGPL 3 is not compatible to the GPL 2
see http://www.gnu.org/licenses/license-list.html for explanation
Since (as far as I know) nobody else has ever contributed to these files I may be allowed to just apply an older license.
You may consider this as a dual-licensing and may use and optionally replicate the older files under GPL 3.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6952 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: lgpl21.txt, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/data/LibraryProvider.java, source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/ai/example/Hanoi.java, source/net/yacy/ai/example/testorder.java, source/net/yacy/ai/greedy/AbstractFinding.java, source/net/yacy/ai/greedy/AbstractModel.java, source/net/yacy/ai/greedy/Agent.java, source/net/yacy/ai/greedy/Asset.java, source/net/yacy/ai/greedy/Attempts.java, source/net/yacy/ai/greedy/Battle.java, source/net/yacy/ai/greedy/Challenge.java, source/net/yacy/ai/greedy/Context.java, source/net/yacy/ai/greedy/ContextFactory.java, source/net/yacy/ai/greedy/Engine.java, source/net/yacy/ai/greedy/Finding.java, source/net/yacy/ai/greedy/Goal.java, source/net/yacy/ai/greedy/Model.java, source/net/yacy/ai/greedy/Role.java, source/net/yacy/ai/greedy/Unirole.java, source/net/yacy/cora/document/Channel.java, source/net/yacy/cora/document/Channels.java, source/net/yacy/cora/document/Hit.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/protocol/HttpConnector.java, source/net/yacy/cora/protocol/ProxySettings.java, source/net/yacy/cora/services/Search.java, source/net/yacy/document/geolocalization/Coordinates.java, source/net/yacy/document/geolocalization/GeonamesLocalization.java, source/net/yacy/document/geolocalization/Localization.java, source/net/yacy/document/geolocalization/Location.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/document/geolocalization/OverarchingLocalization.java, source/net/yacy/document/importer/Importer.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/table/Table.java
Sun Jun 27 23:38:16 CEST 2010
by orbiter
added a configuration page for search heuristics. currently you can switch on there:
- a site-operation heuristic that loads all direct links from a portal page if the site-operator is used
- a direct crawl for search results from scroogle for the given search terms
The configuration page can be found directly beside the network configuration page


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6951 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigHeuristics_p.html, htroot/ConfigHeuristics_p.java, htroot/env/templates/submenuConfig.template, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java
Sun Jun 27 12:45:20 CEST 2010
by orbiter
- added animated visualization for DHT-in and DHT-out in network graphic
- found and fixed a possible memory leak in YaCy internal RSS feed system
- some refactoring in RSS feed mechanisms to make this possible

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6950 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/FeedReader_p.java, htroot/Network.html, htroot/NetworkPicture.java, htroot/rct_p.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/net/yacy/cora/document/Hit.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/visualization/RasterPlotter.java
Sun Jun 27 00:39:27 CEST 2010
by orbiter
- some refactoring for internal RSSFeed (protocol of all actions as seen on status page)
- added dht-out to internal RSSFeed (you can see now messages about distributed indexes on status page)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6948 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.html, htroot/Status.html, htroot/api/feed.java, htroot/terminal_p.html, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/net/yacy/cora/document/RSSFeed.java
Sat Jun 26 12:32:47 CEST 2010
by orbiter
- removed the UPnP library sources from sbbi and added the jar library again. The library was included to get support for fedora releases, but after this time the fact that the sbbi cannot be part of fedora should be re-discussed. If this will still not be possible, then we may integrate the sbbi UPnP package using reflection.
- cleaned uo the code. The new eclipse helios provided new warnings for dead code. This change cleans up most of these warnings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6945 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.xml, htroot/Connections_p.java, htroot/Steering.java, htroot/ViewImage.java, htroot/ViewProfile.java, lib/sbbi-upnplib-1.0.4.jar, source/de/anomic/http/client/Client.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/ai/greedy/Engine.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/kelondro/order/CloneableMapIterator.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/util/AttrSeq.java, source/net/yacy/yacy.java
Fri Jun 25 18:44:57 CEST 2010
by orbiter
added a 'transition feature' that shall lower the barrier to move from g**gle to yacy (yes!):

Here a new concept called 'search heuristics' is introduced. A heuristic is a kind of 'shortcut' to good results in IT, here for good search results. In this case it will be used to get a very transparent way to compare what YaCy is able to produce as search result and what g**gle produces as search result. Here is what your can do now:
- add the phrase 'heuristic:scroogle' to your search query, like 'oil spill heuristic:scroogle' and then a call to scroogle is made to get anonymous search results from g**gle.
- these results are _not_ taken as meta-search results, but are used to instantly feed a crawling and indexing process. This happens very fast, here 20 results from scroogle are taken and loaded all simultanously, parsed and indexed immediately and from the results of the parsed content the search result is feeded, along to the normal p2p search
- when new results from that heuristic (more to come) get part of the search results, then it is verified if such results are redundant to existing (they had been part of the normal YaCy search result anyway) or if they had been completely new to YaCy.
- in the search results the new search results from heuristics are marked with a 'H ++' and search results from heuristics that had been already found by YaCy are marked with a 'H ='. That means:
- you can now see YaCy and Scroogle search results in one result page but you also see that you would not have 'missed' the g**gle results when you would only have used YaCy.

- to make it short: YaCy now subsumes g**gle results. If you use only YaCy, you miss nothing.

to come: a configuration page that let you configure the usage of heuristics and get this feature by default.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6944 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Crawler_p.java, htroot/env/base.css, htroot/env/grafics/heuristic_new.gif, htroot/env/grafics/heuristic_redundant.gif, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/graphics/OSMTile.java
Wed Jun 23 13:19:32 CEST 2010
by orbiter
added a fascinating new way to search _and_ start a web crawl at the same time:
implemented a hint from dulcedo "use site: - operator as crawl start point".
YaCy already was able to search using a site-constraint. This function is now extended with a instant crawling feature.
When you now use the site-operator, then the landing page of the site iand every page that is linked from this page are loaded, indexed and selected for the search result within that search request. When the remote server responds quickly enough, then this process can result in search results during the normal search result preparation .. just in some seconds.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6941 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java
Tue Jun 22 14:28:53 CEST 2010
by orbiter
more abstraction for access of LoaderDispatcher and cache

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6937 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/DictionaryLoader_p.java, htroot/RSSLoader_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/api/util/getpageinfo_p.java, htroot/yacysearch.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/Segment.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/graphics/OSMTile.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/repository/LoaderDispatcher.java
Mon Jun 21 16:54:54 CEST 2010
by orbiter
more abstraction of the htcache when using the LoaderDispatcher:
a cache access shall not made directly to the cache any more, all loading attempts shall use the LoaderDispatcher.
To control the usage of the cache, a enum instance from CrawlProfile.CacheStrategy shall be used.
Some direct loading methods without the usage of a cache strategy have been removed. This affects also the verify-option
of the yacysearch servlet. If there is a 'verify=false' now after this commit this does not necessarily mean that no snippets
are generated. Instead, all snippets that can be retrieved using the cache only are presented. This still means that the search hit was not verified because the snippet was generated using the cache. If a cache-based generation of snippets is not possible, then the verify=false causes that the link is not rejected.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6936 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/RSSLoader_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverCore.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/repository/LoaderDispatcher.java
Thu Jun 17 13:59:40 CEST 2010
by orbiter
- scanned remote search process and enhanced some data structure and synchronizations here and there
- removed concurrency overhead for small number of index normalizations as it happens during remote search
- removed 'load only parseable' constraint for snippet fetch because some resources may not have any url file extension and these had therefore not been parseable and searcheable since they may become parseable after loading when their mime type is known
- this partly fixes some problems with http://forum.yacy-websuche.de/viewtopic.php?p=20300#p20300 but more changes are necessary to get all expected search results

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6926 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/repository/LoaderDispatcher.java
Thu Jun 17 00:16:03 CEST 2010
by orbiter
new license for ai.greedy component: LGPL (nobody else than me modified that code)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6925 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/ai/example/Hanoi.java, source/net/yacy/ai/example/testorder.java, source/net/yacy/ai/greedy/AbstractFinding.java, source/net/yacy/ai/greedy/AbstractModel.java, source/net/yacy/ai/greedy/Agent.java, source/net/yacy/ai/greedy/Asset.java, source/net/yacy/ai/greedy/Attempts.java, source/net/yacy/ai/greedy/Battle.java, source/net/yacy/ai/greedy/Challenge.java, source/net/yacy/ai/greedy/Context.java, source/net/yacy/ai/greedy/ContextFactory.java, source/net/yacy/ai/greedy/Engine.java, source/net/yacy/ai/greedy/Finding.java, source/net/yacy/ai/greedy/Goal.java, source/net/yacy/ai/greedy/Model.java, source/net/yacy/ai/greedy/Role.java, source/net/yacy/ai/greedy/Unirole.java
Tue Jun 15 21:44:05 CEST 2010
by orbiter
better handling of OOM situations

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6918 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Table_API_p.java, htroot/Tables_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/BookmarkDate.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/http/client/Cache.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Stack.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/util/LookAheadIterator.java
Tue Jun 15 12:43:47 CEST 2010
by orbiter
- extended news size from 512 to 1024 characters
- a new news db will be created (news1024.db), the old one (news.db) can be deleted
- peers with too large news payload are not ignored any more (they may have been invisible because they had a too large news payload!)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6917 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, htroot/Blog.java, htroot/Bookmarks.java, htroot/ConfigProfile_p.java, htroot/Crawler_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Wiki.java, htroot/api/bookmarks/posts/add_p.java, htroot/yacysearch.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyPeerActions.java
Tue Jun 01 15:02:11 CEST 2010
by orbiter
- more abstraction (HashMap -> Map)
- more concurrency-awareness (HashMap -> ConcurrentHashMap)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6910 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/MessageSend_p.java, htroot/ViewProfile.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/HeaderFramework.java, source/de/anomic/http/server/RequestHeader.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverAccessTracker.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/document/geolocalization/GeonamesLocalization.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/document/parser/html/CharacterCoding.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/util/Domains.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/ISO639.java, source/net/yacy/kelondro/util/ObjectSpace.java
Mon May 31 02:27:00 CEST 2010
by orbiter
fixed url crawl list display

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6908 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/Balancer.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java
Wed May 26 02:01:16 CEST 2010
by orbiter
redesign of remote proxy settings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6903 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlResults.java, htroot/SettingsAck_p.java, htroot/api/util/getpageinfo_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Switchboard.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/HttpConnector.java, source/net/yacy/cora/protocol/ProxySettings.java, source/net/yacy/yacy.java
Tue May 25 14:54:57 CEST 2010
by orbiter
- added new protocol loader for 'file'-type URLs
- it is now possible to crawl the local file system with an intranet peer
- redesign of URL handling
- refactoring: created LGPLed package cora: 'content retrieval api' which may be used externally by other applications without yacy core elements because it has no dependencies to other parts of yacy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6902 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/Collage.java, htroot/Crawler_p.java, htroot/FeedReader_p.java, htroot/SettingsAck_p.html, htroot/SettingsAck_p.java, htroot/Settings_Crawler.inc, htroot/Settings_p.java, htroot/ViewFile.java, htroot/api/feed.java, htroot/rct_p.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, htroot/yacysearch_location.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/MimeTable.java, source/de/anomic/net/ftpc.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverObjects.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/cora/document/Channel.java, source/net/yacy/cora/document/Channels.java, source/net/yacy/cora/document/Hit.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/Punycode.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/protocol/HttpConnector.java, source/net/yacy/cora/services/Search.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/Idiom.java, source/net/yacy/document/ParserException.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/util/Domains.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/LoaderDispatcher.java, test/de/anomic/yacy/yacyURLTest.java
Fri May 21 10:18:04 CEST 2010
by orbiter
- show up to 10 locations (maps) after search (instead of a max of 5)
- order locations by (primary) population and (secondary) longitude (reverse ordering, both)
- added population from GeoNames, OpenGeoDB does not have that information
- changed default viewpoint of map to (30,15); shows more land and europe in the center

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6893 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.java, htroot/yacysearch_location.html, source/net/yacy/document/geolocalization/Coordinates.java, source/net/yacy/document/geolocalization/GeonamesLocalization.java, source/net/yacy/document/geolocalization/Localization.java, source/net/yacy/document/geolocalization/Location.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/document/geolocalization/OverarchingLocalization.java
Fri May 21 00:05:04 CEST 2010
by orbiter
- fixes to query parameter
- replaced/removed search query attribute (was old style, new is 'query' according to SRU)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6892 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ViewFile.java, htroot/api/util/ynetSearch.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/de/anomic/search/QueryParams.java, source/net/yacy/document/parser/xml/opensearchdescriptionReader.java
Wed May 19 14:53:09 CEST 2010
by orbiter
- added option to localization search to distinguish between a search for a location according to the search word only or for the relation between a web search results and locations found in the metadata fields
- used that to display two layers on map: cities and search result locations
- added many marker grafics for the display of the markers on the map
- some refactoring of the yacy news code plus bugfixes for latest move from Tree to Table data structure

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6889 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blog.java, htroot/Bookmarks.java, htroot/ConfigProfile_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/Crawler_p.java, htroot/Network.java, htroot/News.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewProfile.java, htroot/Wiki.java, htroot/api/bookmarks/posts/add_p.java, htroot/env/grafics/dot_blue.png, htroot/env/grafics/dot_darkgrey.png, htroot/env/grafics/dot_green.png, htroot/env/grafics/dot_lightgrey.png, htroot/env/grafics/dot_red.png, htroot/env/grafics/dot_yellow.png, htroot/env/grafics/marker_blue.png, htroot/env/grafics/marker_darkgrey.png, htroot/env/grafics/marker_green.png, htroot/env/grafics/marker_lightgrey.png, htroot/env/grafics/marker_red.png, htroot/env/grafics/marker_yellow.png, htroot/env/grafics/star_blue.png, htroot/env/grafics/star_darkgrey.png, htroot/env/grafics/star_green.png, htroot/env/grafics/star_lightgrey.png, htroot/env/grafics/star_red.png, htroot/env/grafics/star_yellow.png, htroot/yacysearch.java, htroot/yacysearch_location.html, htroot/yacysearch_location.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyPeerActions.java, source/net/yacy/document/geolocalization/GeonamesLocalization.java, source/net/yacy/document/geolocalization/Localization.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/document/geolocalization/OverarchingLocalization.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/Table.java
Sun May 16 01:49:30 CEST 2010
by orbiter
added another geolocalization data source: GeoNames
- added downloader option in DictionaryLoader
- added generalization (interfaces and overarching localization)
- more abstraction using the libraries

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6879 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/DictionaryLoader_p.html, htroot/DictionaryLoader_p.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearch_location.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/LibraryProvider.java, source/de/anomic/yacy/yacyNewsQueue.java, source/net/yacy/document/geolocalization/Coordinates.java, source/net/yacy/document/geolocalization/GeonamesLocalization.java, source/net/yacy/document/geolocalization/Localization.java, source/net/yacy/document/geolocalization/Location.java, source/net/yacy/document/geolocalization/OpenGeoDBLocalization.java, source/net/yacy/document/geolocalization/OverarchingLocalization.java
Sat May 15 01:50:07 CEST 2010
by orbiter
- fix for caching of OSM tiles. if you want that this fix applies to your peer, please delete the crawl profiles
- fix for initial generation of crawl profiles (one more reason to remove your crawl profiles)
- more String -> byte[] migration
- more logging for cache store/hit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6874 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CacheResource_p.java, htroot/ViewFile.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/data/BookmarkDate.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/http/client/Cache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/OSMTile.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/yacy.java
Fri May 14 20:30:11 CEST 2010
by orbiter
- removed download-limit that can be given for the crawler for non-crawler download tasks. This was necessary because the same procedure was used for other downloads like for the download of dictionary files where a limit is not useful. The limit still stays for the indexer
- migrated the opengeodb downloader to a new version of the opengeodb-dump


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6873 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.html, htroot/DictionaryLoader_p.java, htroot/QuickCrawlLink_p.java, htroot/RSSLoader_p.java, htroot/ViewFile.java, htroot/api/util/getpageinfo_p.java, htroot/yacysearch.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/LibraryProvider.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/http/client/ClientGetMethod.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/graphics/OSMTile.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/document/geolocalization/OpenGeoDB.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/repository/LoaderDispatcher.java
Tue May 11 14:58:05 CEST 2010
by orbiter
added a location search service: using the following servlet/example:
http://localhost:8080/yacysearch_location.kml?query=berlin&maximumTime=2000&maximumRecords=100

This will open any application that can consume kml data (which will probably be google earth) on your computer and displays the search result as positions on a map


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6865 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearch_location.kml, htroot/yacysearch_location.xml, htroot/yacysearchitem.java, htroot/yacysearchitem.xml, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/document/content/RSSMessage.java, source/net/yacy/document/geolocalization/OpenGeoDB.java, source/net/yacy/kelondro/util/DateFormatter.java
Tue May 11 13:14:05 CEST 2010
by orbiter
This is a large bugfix and enhancement commit to support a better location detection for data
- fixes to http file server session handling
- fixes and enhancements to metadata date/time handling
- added dc:publisher metadata field and updated all document parser
- fixed bug in metdata read procedure
- enhanced dublin core and rss parser to understand more fields more properly
- enhanced url selection in case that multiple urls are given in surrogates
- fix for condenser; failure when last word does not end with termination symbol

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6863 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/httpd.mime, htroot/AccessTracker_p.java, htroot/Blacklist_p.java, htroot/Connections_p.java, htroot/Network.java, htroot/PerformanceQueues_p.java, htroot/api/yacydoc.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchitem.xml, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/HeaderFramework.java, source/de/anomic/http/server/RequestHeader.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/Segment.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverHandler.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNetwork.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/RSSMessage.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/geolocalization/OpenGeoDB.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/DateFormatter.java, source/net/yacy/kelondro/util/Domains.java
Sun May 09 00:09:36 CEST 2010
by orbiter
fixed dates in metadata

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6860 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Crawler_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/api/queues_p.java, htroot/rct_p.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/repository/LoaderDispatcher.java
Wed May 05 23:12:58 CEST 2010
by suessthomas
Fixes for (X)HTML compatibility.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6854 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.html, htroot/Connections_p.html, htroot/IndexControlURLs_p.html, htroot/IndexImportOAIPMHList_p.html, htroot/Network.html, htroot/Table_API_p.html, htroot/Tables_p.html, htroot/Trails.html, htroot/api/table_p.html, htroot/sharedBlacklist_p.html, htroot/yacyinteractive.html
Mon May 03 00:20:24 CEST 2010
by orbiter
- fixed resumption token generation for oai-pmh import
- relaxed dublin core parsing: the dc:reference tag may replace dc:identifier if this does not contain a valid url
- parsing of completeRecords number and presentation in the download list of oai import

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6850 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexImportOAIPMHList_p.html, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/document/importer/ResumptionToken.java
Fri Apr 30 16:03:51 CEST 2010
by orbiter
enhanced and fixed OAI-PMH import
- now importing OAI-PMH server list fron two sources
- simultanous import from several servers (even > 2000)
- check buttons on OAI-PMH server list to select multiple servers for import start
- it is possible to select all servers at once for import
- imported XML data is gzipped after import from surrogate reader

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6847 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.properties, htroot/CrawlResults.java, htroot/IndexImportOAIPMHList_p.html, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.html, htroot/IndexImportOAIPMH_p.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/importer/Importer.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/ScoreCluster.java, source/net/yacy/repository/LoaderDispatcher.java
Tue Apr 27 23:47:41 CEST 2010
by orbiter
- cleanup, removed unused imports
- added crawling queue sizes to /api/status_p.xml, syntax same as in queues_p.html
- fixed a bug in queue enumeration that caused a out of bounds exception

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6842 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Crawler_p.html, htroot/Crawler_p.java, htroot/RSSLoader_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/status_p.xml, htroot/yacy/search.java, source/de/anomic/crawler/AbstractImporter.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/dbtest.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/order/StackIterator.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java
Sun Apr 25 23:37:36 CEST 2010
by orbiter
some performance hacks and fixed after reading dump in
http://forum.yacy-websuche.de/viewtopic.php?p=19920#p19920

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6837 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacy/search.java, htroot/yacysearchitem.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/TermSearch.java
Fri Apr 23 11:32:29 CEST 2010
by orbiter
removed finalize methods because of a hint in
http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/memleaks.html#gbyvh

The finalize method prevents that the memory, used by the objects containing the finalize method, is collected and available for the garbage collector. Instead, the memory allocated by such classes are enqueued to a java-internal finalize queue runner. This slows down all operations that uses a lot of object containing finalize methods.

this fix does not remove all finalize method, but such that may be used for throw-away objects that are allocated many times. This should cause a better run-time performance and less OutOfMemoryErrors 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6835 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/http/server/ResponseContainer.java, source/de/anomic/server/serverCore.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/AbstractScraper.java, source/net/yacy/document/parser/html/AbstractTransformer.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/io/ByteCountInputStream.java, source/net/yacy/kelondro/io/ByteCountOutputStream.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/net/yacy/kelondro/util/XMLTables.java
Tue Apr 20 16:47:41 CEST 2010
by orbiter
enhanced remove() operation: in many cases it is not necessary to return the removed object to the called.
for such cases the delete() operation was introduced which is sometimes much cheaper in operation since it does not need to create objects to hold the removed content and it does not need to read those objects.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6824 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/ZURL.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/yacy/yacyNewsDB.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java
Tue Apr 20 15:45:22 CEST 2010
by orbiter
enhanced remove operation in search consequences (which are triggered when the snippet fetch proves that the word has disappeared from the page that was stored in the index)
- no direct deletion of referenced during search (shifted to time after search)
- bundling of all deletions for the references of a single word into one remove operation
- enhanced remove operation by caring that the collection is stored sorted (experimental)
- more String -> byte[] transition for search word lists
- clean up of unused code
- enhanced memory allocation of RowSet Objects (will use a little bit less memory which was wasted before)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6823 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java
Mon Apr 19 18:42:37 CEST 2010
by orbiter
a collection of performance hacks and code cleanup:
- removed usage of URL-Caches which could have been a memory leak
- removed unused classes and methods
- removed not necessary synchronizations
- added synchronization hacks where possible
- fine-tuned crawling speed to prevent IO of balancer
- fixed a bug in IODispatcher that may have caused that no merges were done
- reduced number of parameters in very often called methods (compare methods)
- reduced complexity of data structures of now massively used HandleSet class
- reduction of new String() and getBytes() usage / new methods to support this transition

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6820 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/index/BinSearch.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/order/AbstractOrder.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/Bitfield.java, source/net/yacy/kelondro/order/ByteOrder.java, source/net/yacy/kelondro/order/NaturalOrder.java, source/net/yacy/kelondro/order/StackIterator.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/ByteArray.java
Sun Apr 18 23:55:20 CEST 2010
by orbiter
reduced IO-load and synchronization/blocking
- enhanced the Balancer performance when building new domain stacks using a new Table buffer
- added the new Table buffer BufferedObjectIndex class
- changed order of access to LURL-read (prefereing segment over Crawl Queues) will reduced blocking time on balancer
- fixed PPM setting in Crawler_p servlet (had doubled values)
- reduced synchronization in IndexCell because it is not necessary: reduced blocking during indexing/merging/dumping
- removed did-you-mean cache in IndexCell because that caused too much overhead and more memory usage but was not very useful. This reduced also deadlocks that could be causes when searched are performed during indexing.




git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6819 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java
Thu Apr 15 16:19:29 CEST 2010
by orbiter
enhanced object hash computation 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6816 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceVars.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/util/ByteArray.java
Thu Apr 15 15:22:59 CEST 2010
by orbiter
continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 and continued in SVN 6790
The result should be a less usage of new String() and less memory usage (since a String-encapsulated byte[] has 40 bytes overhead)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6815 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlResults.java, htroot/IndexControlRWIs_p.java, htroot/RSSLoader_p.java, htroot/api/timeline.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/AbstractImporter.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PartitionScheme.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/dht/VerticalWordPartitionScheme.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/net/yacy/document/Condenser.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/image/ImageReferenceVars.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/navigation/NavigationReference.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/IndexReader.java, source/net/yacy/kelondro/rwi/Reference.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/TermSearch.java, source/net/yacy/kelondro/util/CompressedHashMap.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/MapTools.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/yacy.java
Thu Apr 15 00:17:18 CEST 2010
by orbiter
removed usage of temporary files: causes too much IO

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6813 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/client/Client.java, source/net/yacy/document/Document.java, source/net/yacy/document/Idiom.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/zipParser.java
Wed Apr 14 01:29:55 CEST 2010
by orbiter
performance hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6807 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverCore.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/util/EventTracker.java, source/net/yacy/kelondro/workflow/AbstractBusyThread.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java
Tue Apr 13 03:16:09 CEST 2010
by low012
*) changes for better code readability

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6797 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blog.java, htroot/ConfigLanguage_p.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/data/LibraryProvider.java, source/de/anomic/data/MimeTable.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/URLLicense.java, source/de/anomic/data/WorkTables.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/listManager.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java
Thu Apr 08 02:11:32 CEST 2010
by orbiter
continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6790 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/Crawler_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/QuickCrawlLink_p.java, htroot/RSSLoader_p.java, htroot/SettingsAck_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/WebStructurePicture_p.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/queues_p.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/cytag.java, htroot/rct_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/RequestHeader.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/search/blockrank/RCIEvaluation.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/dbtest.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/image/ImageReferenceRow.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadata.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/ObjectArrayCache.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/ByteOrder.java, source/net/yacy/kelondro/order/NaturalOrder.java, source/net/yacy/kelondro/table/RecordStack.java, source/net/yacy/kelondro/table/Records.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/yacy.java
Fri Mar 26 19:33:20 CET 2010
by orbiter
redesign of reference hash (URL-hash) parameter hand-over:
pass value as byte[], not as String. This should cause that less
byte[] <-> String conversions are made during time-critical tasks.
This redesign is not yet complete, more to come ..

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6775 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlResults.java, htroot/Crawler_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/QuickCrawlLink_p.java, htroot/api/yacydoc.java, htroot/yacy/crawlReceipt.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadata.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java
Tue Mar 23 11:17:28 CET 2010
by orbiter
using a Pattern object with precompiled regular expressions to apply must-match constraints to search results: should speed up pre-sorting of search results and should cause richer search result sets

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6762 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearchtrailer.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java
Thu Mar 11 16:43:06 CET 2010
by orbiter
- (almost) fixed FTP crawler
- integrated/fixed SMB crawler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6742 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/IndexControlURLs_p.java, htroot/ViewFile.java, htroot/api/util/getpageinfo_p.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/net/ftpc.java, source/de/anomic/search/Switchboard.java, source/net/yacy/repository/LoaderDispatcher.java
Tue Mar 09 12:31:15 CET 2010
by orbiter
added a robots.txt monitor in the crawler monitor submenu

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6733 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Table_RobotsTxt_p.html, htroot/Table_RobotsTxt_p.java, htroot/Tables_p.java, htroot/api/table_p.html, htroot/api/table_p.java, htroot/api/table_p.xml, htroot/env/templates/submenuCrawlMonitor.template, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Tables.java
Sun Mar 07 22:19:23 CET 2010
by orbiter
after frequent occurrences of 100% CPU usages and permanent blockings I try to disable a function in a method that may cause the problem when calling an external library (apache http client 3.x). The thread dump that shows the problem is attached here.

	at java.lang.StringCoding.encode(StringCoding.java:266)
	at java.lang.String.getBytes(String.java:946)
	at org.apache.commons.httpclient.util.EncodingUtil.getAsciiBytes(EncodingUtil.java:237)
	at org.apache.commons.httpclient.methods.multipart.Part.sendDispositionHeader(Part.java:220)
	at org.apache.commons.httpclient.methods.multipart.Part.send(Part.java:308)
	at org.apache.commons.httpclient.methods.multipart.Part.sendParts(Part.java:385)
	at org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity.writeRequest(MultipartRequestEntity.java:164)
	at de.anomic.http.client.Client.zipRequest(Client.java:364)
	at de.anomic.http.client.Client.POST(Client.java:339)
	at de.anomic.yacy.yacyClient.wput(yacyClient.java:285)
	at de.anomic.yacy.yacyClient.transferURL(yacyClient.java:1053)
	at de.anomic.yacy.yacyClient.transferIndex(yacyClient.java:942)
	at de.anomic.yacy.dht.Transmission$Chunk.transmit(Transmission.java:200)
	at de.anomic.yacy.dht.Dispatcher.storeDocumentIndex(Dispatcher.java:397)
	at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:103)
	at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:66)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:637)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6726 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/client/Client.java
Fri Mar 05 22:25:49 CET 2010
by orbiter
- added a new Wiki Namespace Navigator
- some redesign of Navigator data structures

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6716 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/search.java, htroot/yacysearch.html, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, htroot/yacysearchtrailer.xml, source/de/anomic/data/list/ListAccumulator.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/Navigator.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/net/yacy/kelondro/blob/BEncodedHeap.java
Fri Mar 05 16:41:15 CET 2010
by orbiter
- fixed some bugs in Table viewer
- added 'select all' feature in Tables_p
- enhanced ViewFile.html: has now an input field to load arbitrary resources from the web and analyze them (!!!)
- included the ViewFile servlet into the Index Administration menu
- show in ViewFile if ressource is in url-db and/or in Web cache
- bugfixes to BEncodedHeap and Tables management

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6713 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Table_API_p.html, htroot/Tables_p.html, htroot/Tables_p.java, htroot/ViewFile.html, htroot/ViewFile.java, htroot/env/templates/submenuIndexControl.template, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Tables.java
Thu Mar 04 12:58:07 CET 2010
by orbiter
- fixed a bug in robots.txt parser
- moved storage of robots.txt entries to WorkTables, so it is now possible to browse the robots entries with the table browser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6710 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/robotsParser.java, source/de/anomic/data/WorkTables.java, source/de/anomic/search/Switchboard.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Tables.java
Thu Mar 04 00:32:56 CET 2010
by orbiter
- added parsing of robots meta-tag in html headers to detect a noindexing request
- added evaluation and indexing prevention in case that a noindexing is given in a html file

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6709 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java
Sat Feb 27 04:04:11 CET 2010
by mikeworks
Renamed, removed and added license information for each jar archive in external lib folder

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6704 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: lib/J7Zip-modified.License, lib/activation.License, lib/bcmail-jdk15-145.License, lib/bcprov-jdk15-145.License, lib/bzip2.License, lib/commons-httpclient-3.1.License, lib/log4j-1.2.15.License, lib/mysql-connector-java-5.1.12-bin.License, lib/poi-3.6-20091214.License, lib/poi-scratchpad-3.6-20091214.License, lib/xerces.License
Wed Feb 24 14:53:55 CET 2010
by orbiter
changed way how number of search requests are counted:
so far only search requests at the remote search interface had been counted.
This was done to protect the privacy of searchers, because counting was not done and published at the own search interface.
This caused that no search requests of robinson peers had been counted, becuase they cannot be counted at remote peer.
This change introduces a distinction of locally done search requests at the local search interface from search requests that are on the local interface but had been submitted from a remote IP without authentication.
Now 3 counters are maintained:
- partial count of remote searches
- total count of local searches on robinson peers from non-authenticated clients
- total count of local searches on robinson peers from localhost or authenticated clients
In the global statistic of search requests now the first two counters of the three cases are added
Because we habe a large number of robinson peers with a large number of remote non-authenticated requests the statistic should show at least three times of the number of search requests.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6696 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.html, htroot/Network.java, htroot/Network.xml, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/search/Switchboard.java
Mon Feb 22 21:14:16 CET 2010
by orbiter
- moved test data from /bin to /test/words
- refactoring of stopYACY.sh by introduction of /bin/apicall which is able to call any api file with attached authorization

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6691 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: bin/apicall.sh, bin/searchtestmulti.sh, stopYACY.sh, test/words/searchtest.words, test/words/searchtest.words.aa, test/words/searchtest.words.ab, test/words/searchtest.words.ac, test/words/searchtest.words.ad, test/words/searchtest.words.ae, test/words/searchtest.words.af, test/words/searchtest.words.ag, test/words/searchtest.words.ah, test/words/searchtest.words.ai, test/words/searchtest.words.aj, updateYACY.sh
Thu Feb 04 12:26:23 CET 2010
by orbiter
refactoring of tables data organisation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6644 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigPortal.java, htroot/CrawlMonitorRemoteStart.java, htroot/Crawler_p.java, htroot/IndexCreateDomainCrawl_p.java, htroot/Table_API_p.java, htroot/Tables_p.java, source/de/anomic/data/WorkTables.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/util/LookAheadIterator.java
Mon Feb 01 23:18:56 CET 2010
by orbiter
refactoring of bookmarks: there is a big performance problem in the bookmarks code and furthermore the bookmarks
will loose its leading role for the re-crawl funtion when the new api tables will work. To be prepared for a replacement
of such functions the bookmark class is re-organised.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6637 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/Network.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/xbel/xbel.java, source/de/anomic/data/BookmarkDate.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/blob/MapDataMining.java
Fri Jan 29 19:02:09 CET 2010
by orbiter
- added Tables abstraction in data.Tables.java
fix for
http://forum.yacy-websuche.de/viewtopic.php?p=18910#p18910
http://forum.yacy-websuche.de/viewtopic.php?p=18894#p18894
http://forum.yacy-websuche.de/viewtopic.php?p=18814#p18814


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6631 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigPortal.java, htroot/Crawler_p.java, htroot/Tables_p.java, source/de/anomic/data/Tables.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverSwitch.java, source/net/yacy/kelondro/blob/BEncodedHeapArray.java, source/net/yacy/kelondro/blob/HeapReader.java
Fri Jan 29 16:59:24 CET 2010
by orbiter
misc NPE check

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6630 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlURLs_p.java, htroot/yacy/transferURL.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java
Thu Jan 21 23:07:32 CET 2010
by orbiter
update to yacy/ai (just more testing)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6607 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/ai/example/Hanoi.java, source/net/yacy/ai/greedy/AbstractFinding.java, source/net/yacy/ai/greedy/Agent.java, source/net/yacy/ai/greedy/Battle.java, source/net/yacy/ai/greedy/Context.java, source/net/yacy/ai/greedy/Engine.java, source/net/yacy/ai/greedy/Goal.java, source/net/yacy/ai/greedy/Unirole.java
Thu Jan 21 23:06:03 CET 2010
by orbiter
added an api recording function: it shall be possible to record
all operations on YaCy in a database that should make it possible
1) to re-create a setting on fresh peers
2) to transmit a setting from one peer to another
3) to re-create crawl starts after a complete deletion of the index
This functionality will also support
4) scheduled re-crawls (new implementation)
To implement this, a new database structure has been crated that stores maps into blob heaps. to encode maps the b-encoding technique was used (this is the same encoding that torrent files use)
- added a b-encoder
- enhanced the b-decoder
- added a b-encoded map heap data structure
- added a table organisation based on b-encoded heaps
- added a servlet to maintain such tables (see Tables_p.html)
- integrated the servlet into the Advanced Settings menu
- added an api recording based on the new tables

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6606 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigPortal.java, htroot/Crawler_p.java, htroot/Settings_p.html, htroot/Tables_p.html, htroot/Tables_p.java, htroot/env/templates/header.template, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/net/ftpc.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/BEncodedHeapArray.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/BEncoder.java, source/net/yacy/yacy.java
Wed Jan 13 17:21:37 CET 2010
by orbiter
- added BEncoder class
- added BEncodedHeap class that encodes B data structures and stores that to a heap
- refactoring of MapView, this is now named MapHeap to fit into the naming scheme of the BEncodedHeap

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6579 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/http/client/Cache.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/BEncoder.java, source/net/yacy/yacy.java
Wed Jan 13 01:23:07 CET 2010
by orbiter
more PMD enhancements

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6576 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/Crawler_p.java, htroot/opensearchdescription.java, htroot/processing/domaingraph/applet/domaingraph.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/de/anomic/crawler/robotsParser.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/HeaderFramework.java, source/de/anomic/net/natLib.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacySeedDB.java, source/net/sbbi/upnp/HttpResponse.java, source/net/yacy/document/Document.java, source/net/yacy/kelondro/data/meta/DigestURI.java
Tue Jan 12 21:53:19 CET 2010
by orbiter
more PMD recommendations

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6572 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/ConfigAppearance_p.java, htroot/CrawlProfileEditor_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/Wiki.java, htroot/yacy/message.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/server/RequestHeader.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/Switchboard.java, source/net/sbbi/upnp/ServicesEventing.java, source/net/sbbi/upnp/messages/ActionMessage.java, source/net/sbbi/upnp/messages/ActionResponse.java, source/net/sbbi/upnp/services/ISO8601Date.java, source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/kelondro/io/AbstractWriter.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java, source/net/yacy/kelondro/util/DateFormatter.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/MapTools.java, source/net/yacy/kelondro/util/ScoreCluster.java, source/net/yacy/migration.java, source/net/yacy/repository/Blacklist.java
Tue Jan 12 11:05:28 CET 2010
by orbiter
renamed the servlet WatchCrawler_p to Crawler_p
this was done because that servlet may be used for wget/cronjob
triggered crawl starts and it appears to be confusing that the
name of the crawl start servlet looks like a pure monitoring tool.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6568 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigPHPBB3Search.html, htroot/ConfigWikiSearch.html, htroot/CrawlStart_p.html, htroot/Crawler_p.html, htroot/Crawler_p.java, htroot/Status.html, htroot/env/base.css, htroot/env/templates/header.template, htroot/env/templates/submenuCrawlMonitor.template, htroot/js/Crawler.js, htroot/js/IndexCreate.js, locales/de.lng, locales/fr.lng, source/de/anomic/http/server/HTTPDFileHandler.java
Mon Jan 11 23:18:38 CET 2010
by orbiter
more PMD recommendations

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6567 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/server/serverObjects.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyUpdateLocation.java, source/net/sbbi/upnp/ServicesEventing.java, source/net/sbbi/upnp/services/UPNPService.java, source/net/yacy/document/content/dao/ImportDump.java, source/net/yacy/yacy.java
Mon Jan 11 00:09:48 CET 2010
by orbiter
applied code changes that are recommended by PMD

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6563 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/CrawlProfileEditor_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/PerformanceQueues_p.java, htroot/Ranking_p.java, htroot/SettingsAck_p.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/util/ynetSearch.java, htroot/api/yacydoc.java, htroot/imagetest.java, htroot/index.java, htroot/processing/domaingraph/applet/domaingraph.java, htroot/yacy/message.java, htroot/yacy/transfer.java, htroot/yacysearch.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/URLLicense.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiCode.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/HeaderFramework.java, source/de/anomic/http/server/RequestHeader.java, source/de/anomic/net/ftpc.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverAccessTracker.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/tools/CryptoLib.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.java, source/de/anomic/yacy/yacyBuildProperties.java, source/de/anomic/yacy/yacyVersion.java, source/net/sbbi/upnp/DiscoveryAdvertisement.java, source/net/sbbi/upnp/DiscoveryListener.java, source/net/sbbi/upnp/HttpResponse.java, source/net/sbbi/upnp/JXPathParser.java, source/net/sbbi/upnp/ServiceEventMessageParser.java, source/net/sbbi/upnp/ServiceEventSubscription.java, source/net/sbbi/upnp/impls/InternetGatewayDevice.java, source/net/sbbi/upnp/messages/ActionMessage.java, source/net/sbbi/upnp/messages/ActionMessageResponseParser.java, source/net/sbbi/upnp/messages/ActionResponse.java, source/net/sbbi/upnp/messages/StateVariableMessage.java, source/net/sbbi/upnp/messages/UPNPMessageFactory.java, source/net/sbbi/upnp/services/ServiceStateVariable.java, source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/ai/greedy/AbstractFinding.java, source/net/yacy/ai/greedy/Agent.java, source/net/yacy/ai/greedy/Asset.java, source/net/yacy/ai/greedy/Challenge.java, source/net/yacy/ai/greedy/Context.java, source/net/yacy/ai/greedy/ContextFactory.java, source/net/yacy/ai/greedy/Engine.java, source/net/yacy/dbtest.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/geolocalization/Coordinates.java, source/net/yacy/document/geolocalization/OpenGeoDB.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHReader.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/document/language/LanguageStatisticsHolder.java, source/net/yacy/document/parser/html/ContentTransformer.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/xml/ODContentHandler.java, source/net/yacy/document/parser/xml/ODMetaHandler.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/Stack.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/ConcurrentARC.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSpaceExceededException.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/order/NaturalOrder.java, source/net/yacy/kelondro/order/StackIterator.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/TermSearch.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/RecordStack.java, source/net/yacy/kelondro/table/Records.java, source/net/yacy/kelondro/util/AttrSeq.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/ByteArray.java, source/net/yacy/kelondro/util/CompressedHashMap.java, source/net/yacy/kelondro/util/DateFormatter.java, source/net/yacy/kelondro/util/DiskSpace.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/OS.java, source/net/yacy/kelondro/util/ScoreCluster.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/kelondro/workflow/AbstractThread.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java, source/net/yacy/repository/Blacklist.java, source/org/apache/tools/tar/TarEntry.java
Sun Jan 10 02:40:26 CET 2010
by orbiter
added some modifications recommended by PMD for better performance

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6560 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/processing/domaingraph/applet/domaingraph.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/Switchboard.java, source/net/yacy/ai/greedy/AbstractFinding.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/ConcurrentARC.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/index/RowSpaceExceededException.java, source/net/yacy/kelondro/index/SimpleARC.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/util/FileUtils.java
Sun Jan 10 01:10:43 CET 2010
by orbiter
some redesign of the main menu structure:
- moved all index generation servlets to it's own main menu item, including proxy indexing
- removed external index import because this operation is not recommended any more. Joining an index can simply be done by moving the index files from one peer to the other peer; they will be merged automatically
- fix to prevent endless loops when disconnecting http sessions
- fix to prevent application of bad blacklist entries that can cause a 'Dangling meta character' exception

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6558 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigPHPBB3Search.html, htroot/ConfigWikiSearch.html, htroot/CrawlProfileEditor_p.html, htroot/IndexCreateLoaderQueue_p.html, htroot/IndexCreateParserErrors_p.html, htroot/IndexCreateWWWGlobalQueue_p.html, htroot/IndexCreateWWWLocalQueue_p.html, htroot/IndexCreateWWWRemoteQueue_p.html, htroot/ProxyIndexingMonitor_p.html, htroot/WatchCrawler_p.html, htroot/WatchCrawler_p.java, htroot/env/templates/header.template, htroot/env/templates/submenuCrawlMonitor.template, htroot/env/templates/submenuIndexControl.template, htroot/env/templates/submenuIndexCreate.template, htroot/env/templates/submenuPortalIntegration.template, source/de/anomic/server/serverCore.java, source/net/yacy/repository/Blacklist.java
Sat Jan 09 01:08:16 CET 2010
by orbiter
fix for http://forum.yacy-websuche.de/viewtopic.php?p=18500#p18500
- catch uncatched OOM
- less wasting of memory

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6555 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/ZURL.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/ObjectArrayCache.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/rwi/AbstractBufferedIndex.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java
Sun Jan 03 23:51:14 CET 2010
by orbiter
Added new classes for the implementation of concurrent greedy algorithms.
These classes can be used to produce an abstract worker process that can be used for common problems in artificial intelligence, such as game playing and problem solving. These classes will be used as abstraction layer for a new search process in YaCy. These classes had been created while searching for an abstraction of the current search process. It turned out that the abstraction of the YaCy search process is also an abstraction for problems in artificial intelligence and therefore the classes had been designed in such a way that it covers not only the YaCy-specific problem but also the more generic problems in ai. To test the classes they had been used in a ConnectFour implementation (game playing). 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6545 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/ai/greedy/AbstractFinding.java, source/net/yacy/ai/greedy/AbstractModel.java, source/net/yacy/ai/greedy/Agent.java, source/net/yacy/ai/greedy/Asset.java, source/net/yacy/ai/greedy/Attempts.java, source/net/yacy/ai/greedy/Battle.java, source/net/yacy/ai/greedy/Challenge.java, source/net/yacy/ai/greedy/Context.java, source/net/yacy/ai/greedy/ContextFactory.java, source/net/yacy/ai/greedy/Engine.java, source/net/yacy/ai/greedy/Finding.java, source/net/yacy/ai/greedy/Goal.java, source/net/yacy/ai/greedy/Model.java, source/net/yacy/ai/greedy/Role.java
Mon Dec 28 13:16:40 CET 2009
by low012
*) adding more SVN properties
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6540 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadata.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/navigation/NavigationReference.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceFactory.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/data/word/WordReference.java, source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java
Thu Dec 10 00:27:26 CET 2009
by orbiter
added extensive memory protection logic to avoid out of memory errors that may be caused by the RowCollection memory allocation function

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6521 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAccounts_p.java, htroot/CrawlProfileEditor_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/News.java, htroot/User.java, htroot/ViewProfile.java, htroot/api/timeline.java, htroot/yacy/transferRWI.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/http/client/Cache.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/dbtest.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapView.java, source/net/yacy/kelondro/blob/Stack.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/ObjectArrayCache.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/index/RowSpaceExceededException.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/TermSearch.java, source/net/yacy/kelondro/table/Records.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/AttrSeq.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/DateFormatter.java, source/net/yacy/yacy.java
Tue Dec 08 15:25:51 CET 2009
by orbiter
- more configuration for profiling graph (number of events)
- more logging for a shutdown: print reason and accessing IP into log


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6520 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigUpdate_p.java, htroot/PerformanceSearch_p.java, htroot/Steering.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/graphics/ProfilingGraph.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/util/EventTracker.java, source/net/yacy/kelondro/workflow/AbstractBlockingThread.java, source/net/yacy/kelondro/workflow/AbstractBusyThread.java, source/net/yacy/kelondro/workflow/AbstractThread.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/kelondro/workflow/WorkflowThread.java, source/net/yacy/yacy.java
Wed Dec 02 15:01:19 CET 2009
by orbiter
preset of proper HashMap dimensions: should prevent re-hashing and increase performance

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6511 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/api/bookmarks/posts/add_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/yacy/dht/Dispatcher.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/parser/html/CharacterCoding.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/ObjectArrayCache.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/rwi/TermSearch.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/Domains.java, source/net/yacy/kelondro/util/ISO639.java
Wed Dec 02 01:37:59 CET 2009
by orbiter
replaced _all_ size() == 0 with isEmpty() and all size() > 0 with !isEmpty(). The isEmpty() method is much faster in some cases, especially when used to access badly balanced hashtables where an size() operation becomes a large iteration.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6510 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/Blacklist_p.java, htroot/ConfigUpdate_p.java, htroot/IndexControlRWIs_p.java, htroot/News.java, htroot/Threaddump_p.java, htroot/api/feed.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ExternalIndexImporter.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/MimeTable.java, source/de/anomic/data/listManager.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverAccessTracker.java, source/de/anomic/tools/disorderHeap.java, source/de/anomic/tools/disorderSet.java, source/de/anomic/tools/enumerateFiles.java, source/de/anomic/tools/loaderCore.java, source/de/anomic/tools/nxTools.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/dbtest.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ContentTransformer.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/xml/RSSFeed.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapView.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/ObjectArrayCache.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/order/StackIterator.java, source/net/yacy/kelondro/rwi/AbstractBufferedIndex.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/TermSearch.java, source/net/yacy/kelondro/table/RecordStack.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/MemoryTracker.java, source/net/yacy/kelondro/util/ObjectSpace.java, source/net/yacy/kelondro/util/ScoreCluster.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/kelondro/util/SortStack.java, source/net/yacy/kelondro/util/SortStore.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/yacy.java
Tue Nov 24 12:13:11 CET 2009
by orbiter
- some refactoring in workflow
- some refactoring in search process
- fixed image search for json and rss output
- search navigation on bottom of search result page in cases where there are more than 6 results on page
- fixes for number of displayed documents
- disabled pseudostemming

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6504 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAccounts_p.html, htroot/IndexControlRWIs_p.java, htroot/env/base.css, htroot/yacy/search.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchitem.xml, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/Condenser.java, source/net/yacy/kelondro/util/SortStack.java, source/net/yacy/kelondro/workflow/BlockingThread.java, source/net/yacy/kelondro/workflow/InstantBlockingThread.java, source/net/yacy/kelondro/workflow/WorkflowJob.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java
Mon Nov 23 17:10:50 CET 2009
by orbiter
added json and rss output for image search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6503 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.rss, htroot/yacysearchitem.html, htroot/yacysearchitem.java, htroot/yacysearchitem.json, htroot/yacysearchitem.xml, source/de/anomic/data/MimeTable.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/MediaSnippet.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/rssParser.java
Fri Nov 20 15:35:33 CET 2009
by orbiter
- better synchronization in SortStack
- better ThreadGroup organization
- less worker threads for media search (64 was too much...)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6497 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearchitem.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/server/serverCore.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/kelondro/util/SortStack.java, source/net/yacy/kelondro/util/SortStore.java
Fri Nov 20 13:11:56 CET 2009
by orbiter
- fixes for missing or bad hashCode computation
- fixes for bad equals() methods that had not been used by hash maps and therefore some classes did not work as objects in hash maps.
- this may also affect some cases where double-checks should have been, but did not work.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6495 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/client/ConnectionInfo.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/document/geolocalization/Coordinates.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/order/AbstractOrder.java, source/net/yacy/kelondro/order/Bitfield.java, source/net/yacy/kelondro/order/Order.java, source/net/yacy/kelondro/rwi/Reference.java, source/net/yacy/kelondro/rwi/ReferenceContainerOrder.java, source/net/yacy/kelondro/table/Records.java, source/net/yacy/kelondro/util/SortStack.java, source/net/yacy/kelondro/util/SortStore.java, source/org/apache/tools/tar/TarEntry.java
Fri Nov 20 04:30:48 CET 2009
by orbiter
added comparator and more fixes for SortStack/SortStore

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6494 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/net/yacy/document/geolocalization/Coordinates.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/table/Records.java, source/net/yacy/kelondro/util/SortStack.java, source/net/yacy/kelondro/util/SortStore.java
Thu Nov 19 14:49:28 CET 2009
by orbiter
Better implementation of SortStack and SortStore and adoptions in all using classes to implement the necessary Comparable interface and hash code computation.
The better SortStack performance affects crawling and image search speed and quality.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6492 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/Reference.java, source/net/yacy/kelondro/util/SortStack.java, source/net/yacy/kelondro/util/SortStore.java
Thu Nov 19 00:56:05 CET 2009
by orbiter
enhanced image search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6489 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Ranking_p.java, htroot/env/base.css, htroot/index.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/search/ContentDomain.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/net/yacy/document/Condenser.java, source/net/yacy/kelondro/util/SortStack.java, source/net/yacy/kelondro/util/SortStore.java
Wed Nov 18 17:03:28 CET 2009
by orbiter
some performance hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6488 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexCreateParserErrors_p.java, htroot/yacy/transferRWI.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HeaderFramework.java, source/de/anomic/http/server/RobotsTxtConfig.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/rssParser.java
Tue Nov 17 00:00:54 CET 2009
by orbiter
- avoid unnecessary host name lookups for localhost
- avoid unnecessary reverse domain name lookups for remote access

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6481 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigBasic.java, htroot/Status.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/net/UPnP.java, source/de/anomic/net/ftpc.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSwitch.java, source/net/yacy/kelondro/io/CachedFileWriter.java, source/net/yacy/kelondro/util/Domains.java, source/net/yacy/kelondro/util/SortStack.java
Sun Nov 08 23:58:57 CET 2009
by orbiter
refactoring (reason: get more abstraction to use the blacklist class; for integration in other servlets)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6471 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/Blacklist_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/IndexControlRWIs_p.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/sharedBlacklist_p.java, htroot/yacy/list.java, source/de/anomic/data/listManager.java, source/de/anomic/data/translator.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/migration.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/BlacklistFile.java
Sat Nov 07 00:15:20 CET 2009
by orbiter
possible fix for lost crawl profile handles: clean-up job did wrong measurement to see if crawl is still running.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6465 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexCreateLoaderQueue_p.java, htroot/PerformanceQueues_p.java, htroot/Status.java, htroot/WatchCrawler_p.java, htroot/api/queues_p.java, htroot/imagetest.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/net/whois.java, source/de/anomic/search/Switchboard.java
Thu Nov 05 21:28:37 CET 2009
by orbiter
added about 450 replacements for printStackTrace() methods to pipe such traces into the log at DATA/LOG/

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6458 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CrawlResults.java, htroot/FeedReader_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexImportOAIPMH_p.java, htroot/IndexImportWikimedia_p.java, htroot/IndexImport_p.java, htroot/News.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/ViewLog_p.java, htroot/WatchCrawler_p.java, htroot/WebStructurePicture_p.java, htroot/api/yacydoc.java, htroot/imagetest.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/query.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/ExternalIndexImporter.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/data/LibraryProvider.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/userDB.java, source/de/anomic/http/client/AcceptEverythingSSLProtcolSocketFactory.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/http/server/servlets/transferURL.java, source/de/anomic/net/ftpc.java, source/de/anomic/net/whois.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/search/blockrank/RCIEvaluation.java, source/de/anomic/server/serverAccessTracker.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/graphics/OSMTile.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/dbtest.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/DatabaseConnection.java, source/net/yacy/document/content/dao/ImportDump.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/geolocalization/OpenGeoDB.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/parser/html/ContentTransformer.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/icoParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/xml/ODContentHandler.java, source/net/yacy/document/parser/xml/RSSReader.java, source/net/yacy/document/parser/xml/opensearchdescriptionReader.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapView.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/io/CachedFileWriter.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java, source/net/yacy/kelondro/logging/Log.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/RecordStack.java, source/net/yacy/kelondro/table/Records.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/AttrSeq.java, source/net/yacy/kelondro/util/ConsoleInterface.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/MemoryTracker.java, source/net/yacy/kelondro/util/OS.java, source/net/yacy/kelondro/workflow/InstantBlockingThread.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java, source/net/yacy/migration.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/visualization/AnimGifEncoder.java, source/net/yacy/visualization/Captcha.java, source/net/yacy/visualization/RasterPlotter.java, source/net/yacy/yacy.java
Tue Nov 03 23:47:53 CET 2009
by low012
*) minor changes mainly in comments
*) added svn:keyword settings for several files

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6451 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/kelondro/order/AbstractOrder.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/Bitfield.java, source/net/yacy/kelondro/order/ByteOrder.java, source/net/yacy/kelondro/order/CloneableIterator.java, source/net/yacy/kelondro/order/CloneableMapIterator.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/order/MicroDate.java, source/net/yacy/kelondro/order/NaturalOrder.java, source/net/yacy/kelondro/order/Order.java, source/net/yacy/kelondro/order/RotateIterator.java, source/net/yacy/kelondro/order/StackIterator.java
Tue Nov 03 21:10:59 CET 2009
by orbiter
- added a csv "comma separated values" parser to parse OAI-PMH sources from
  http://roar.eprints.org/index.php?action=csv
- integrated the csv parser into the crawlers parser list
- added an extension to the OAI-PMH import function to download and show the roar csv file using the csv parser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6448 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexImportOAIPMHList_p.html, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.html, htroot/IndexImportOAIPMH_p.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/parser/csvParser.java
Mon Nov 02 12:53:14 CET 2009
by orbiter
- added function to OAI-PMH reader that can pull all records from a server using an evaluation of the resumption token to get URL to retrieve remaining records
- added monitoring for retrieved records

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6444 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlResults.html, htroot/IndexImportOAIPMH_p.html, htroot/IndexImportOAIPMH_p.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/retrieval/EventOrigin.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHReader.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/kelondro/data/word/Word.java
Mon Nov 02 01:14:14 CET 2009
by orbiter
very first working version of oai-pmh importer: if given the right url, the importer can read and index listRecord xml files and calculate the right resumptionURL which is then given as next default start point for the importer url input.
no automatic harvesting by now, this will be done later

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6443 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexImportOAIPMH_p.html, htroot/IndexImportOAIPMH_p.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/importer/Importer.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/ResumptionToken.java
Sat Oct 31 12:58:06 CET 2009
by orbiter
- some redesign in UI menu structure to make room for new 'Content Integration' main menu containing import servlets for Wikimedia Dumps, phpbb3 forum imports and OAI-PMH imports
- extended the OAI-PMH test applet and integrated it into the menu. Does still not import OAI-PMH records, but shows that it is able to read and parse this data
- some redesign in ZURL storage: refactoring of access methods, better concurrency, less synchronization
- added a limitation to the LURL metadata database table cache to 20 million entries: this cache was until now not limited and only limited by the available RAM which may have caused a memory-leak-like behavior.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6440 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigLiveSearch.html, htroot/ConfigPHPBB3Search.html, htroot/ConfigPortal.html, htroot/ConfigSearchBox.html, htroot/ConfigWikiSearch.html, htroot/ContentIntegrationPHPBB3_p.html, htroot/IndexCreateParserErrors_p.java, htroot/IndexImportOAIPMH_p.html, htroot/IndexImportOAIPMH_p.java, htroot/IndexImportWikimedia_p.html, htroot/WatchCrawler_p.java, htroot/env/templates/header.template, htroot/env/templates/submenuContentIntegration.template, htroot/env/templates/submenuIndexCreate.template, htroot/env/templates/submenuPortalIntegration.template, htroot/yacy/crawlReceipt.java, htroot/yacy/urls.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/importer/ResumptionTokenReader.java, source/net/yacy/kelondro/blob/MapView.java, source/net/yacy/kelondro/index/Cache.java
Fri Oct 23 00:38:04 CEST 2009
by orbiter
- fix for xls parser
- fix for image parser
- temporary integration of images as document types in the crawler and indexer for testing of the image parser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6435 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/xlsParser.java
Wed Oct 21 22:14:30 CEST 2009
by orbiter
- refactoring of blacklists
- refactoring of event origin encoding


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6434 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/CrawlResults.java, htroot/IndexControlRWIs_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/api/blacklists_p.java, htroot/sharedBlacklist_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/retrieval/EventOrigin.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/listManager.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/servlets/transferURL.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacySearch.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/BlacklistFile.java, source/net/yacy/repository/LoaderDispatcher.java
Wed Oct 21 17:12:34 CEST 2009
by orbiter
- more refactoring
- fixed problem with parsers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6433 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/ViewFile.java, htroot/api/util/getpageinfo_p.java, htroot/yacysearch.java, source/de/anomic/crawler/PMHReader.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/repository/LoaderDispatcher.java
Tue Oct 20 00:34:44 CEST 2009
by orbiter
- refactoring
- added new experimental (not-yet-working) image parser
- added new test image

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6431 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Banner.java, htroot/ConfigParser.java, htroot/ConfigUpdate_p.java, htroot/NetworkPicture.java, htroot/PeerLoadPicture.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/SearchEventPicture.java, htroot/ViewImage.java, htroot/WebStructurePicture_p.java, htroot/api/webstructure.java, htroot/cytag.java, htroot/imagetest.java, htroot/osm.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/LibraryProvider.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/graphics/NetworkGraph.java, source/de/anomic/yacy/graphics/OSMTile.java, source/de/anomic/yacy/graphics/ProfilingGraph.java, source/de/anomic/yacy/graphics/WebStructureGraph.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyRelease.java, source/net/yacy/dbtest.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/ImageParser.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/geolocalization/Coordinates.java, source/net/yacy/document/geolocalization/Location.java, source/net/yacy/document/geolocalization/OpenGeoDB.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/icoParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/util/MemoryTracker.java, source/net/yacy/kelondro/util/OS.java, source/net/yacy/visualization/AnimGifEncoder.java, source/net/yacy/visualization/Captcha.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/CircleTool.java, source/net/yacy/visualization/DemoApplet.java, source/net/yacy/visualization/FontGenerator5Pixel.java, source/net/yacy/visualization/FontGenerator8Pixel.java, source/net/yacy/visualization/GraphPlotter.java, source/net/yacy/visualization/PrintTool.java, source/net/yacy/visualization/RasterPlotter.java, source/net/yacy/yacy.java, test/de/anomic/document/ParserTest.java, test/parsertest/image_green_sd.png
Sun Oct 18 03:38:07 CEST 2009
by orbiter
- shifted main classes to new package net.yacy
- fixed some bugs in last commit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6427 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/search/Switchboard.java, source/net/yacy/dbtest.java, source/net/yacy/document/Document.java, source/net/yacy/genpw.java, source/net/yacy/migration.java, source/net/yacy/screenshot.java, source/net/yacy/yacy.java, startYACY.bat, startYACY.sh, startYACY_Win9x.bat, startYACY_debug.bat, startYACY_noconsole_Win9x.bat, stopYACY.bat, stopYACY.command, stopYACY.sh, stopYACY_Win9x.bat
Sun Oct 18 02:53:43 CEST 2009
by orbiter
refactoring of yacy documents and parsers: they depend now only on the kelondro classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6426 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/ConfigParser.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/FeedReader_p.java, htroot/IndexControlRWIs_p.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/feed.java, htroot/api/util/getpageinfo_p.java, htroot/cytag.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/PMHReader.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/wiki/wikiCode.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/http/server/servlets/transferURL.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/serverObjects.java, source/de/anomic/tools/CryptoLib.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/ymage/WebStructureGraph.java, source/de/anomic/ymage/ymageGraph.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Classification.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/Idiom.java, source/net/yacy/document/Parser.java, source/net/yacy/document/ParserException.java, source/net/yacy/document/Phrase.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/RSSMessage.java, source/net/yacy/document/content/dao/Dao.java, source/net/yacy/document/content/dao/DatabaseConnection.java, source/net/yacy/document/content/dao/ImportDump.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/content/file/SurrogateReader.java, source/net/yacy/document/detector/odtDetector.java, source/net/yacy/document/detector/rssDetector.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/document/language/LanguageFilenameFilter.java, source/net/yacy/document/language/LanguageStatistics.java, source/net/yacy/document/language/LanguageStatisticsHolder.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/html/AbstractScraper.java, source/net/yacy/document/parser/html/AbstractTransformer.java, source/net/yacy/document/parser/html/CharacterCoding.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ContentTransformer.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/document/parser/html/Scraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/html/ScraperListener.java, source/net/yacy/document/parser/html/Transformer.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/xml/ODContentHandler.java, source/net/yacy/document/parser/xml/ODMetaHandler.java, source/net/yacy/document/parser/xml/RSSFeed.java, source/net/yacy/document/parser/xml/RSSReader.java, source/net/yacy/document/parser/xml/opensearchdescriptionReader.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/io/CachedFileOutputStream.java, source/net/yacy/kelondro/io/CharBuffer.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, test/de/anomic/document/ParserTest.java, test/de/anomic/document/parser/htmlParserTest.java
Sat Oct 17 02:26:14 CEST 2009
by low012
*) Made sure that only files with appropriate file endings are listed as skin or language files.
*) Introduced protection against directory traversal attacks in configuration servlets for skin and language configuration. Files can only be deleted if they are contained in a list of files which has been read by the servlet first.


Until now it was possible to delete any data on a system YaCy is running on and which can be deleted by the user who's account has been used to start YaCy. Most of the times a user of YaCy is also the owner of the machine the peer is running on, but this might not always be the case and not even the owner of the machine should be able to use YaCy as a replacement for "rm" or "del".

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6423 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/ConfigAppearance_p.html, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.html, htroot/ConfigLanguage_p.java
Wed Oct 14 15:29:09 CEST 2009
by orbiter
- moved httpd.mime to defaults path
- some documentation fixes
- adopted a default setting for the search window: moves css setting to base.css
- some enhancements for the DocumentIndex class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6410 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/httpd.mime, defaults/yacy.init, htroot/env/base.css, htroot/env/templates/header.template, source/de/anomic/document/Classification.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/Segments.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/util/FileUtils.java
Mon Oct 12 23:59:39 CEST 2009
by orbiter
more final modifier


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6407 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/kelondro/blob/MapView.java, source/net/yacy/kelondro/io/AbstractReader.java, source/net/yacy/kelondro/io/AbstractWriter.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/ByteCountInputStream.java, source/net/yacy/kelondro/io/ByteCountOutputStream.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/io/CachedFileWriter.java, source/net/yacy/kelondro/io/RandomAccessIO.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java, source/net/yacy/kelondro/logging/ConsoleOutHandler.java, source/net/yacy/kelondro/logging/GuiHandler.java, source/net/yacy/kelondro/logging/Log.java, source/net/yacy/kelondro/logging/LogParser.java, source/net/yacy/kelondro/logging/LogalizerHandler.java, source/net/yacy/kelondro/logging/MiniLogFormatter.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java
Mon Oct 12 19:37:12 CEST 2009
by orbiter
performance tuning using more final modifiers in the kelondro core

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6402 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/kelondro/index/BinSearch.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/ConcurrentARC.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/ObjectArrayCache.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/index/SimpleARC.java
Mon Oct 12 10:06:35 CEST 2009
by orbiter
fixed a bug in table iterator (did not recognize elements in write buffer)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6401 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java
Sun Oct 11 23:29:18 CEST 2009
by orbiter
refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6400 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigLiveSearch.java, htroot/ConfigNetwork_p.java, htroot/ConfigPHPBB3Search.java, htroot/ConfigParser.java, htroot/ConfigPortal.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/ConfigUpdate_p.java, htroot/ConfigWikiSearch.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/FeedReader_p.java, htroot/Help.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImportWikimedia_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/Performance_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/Trails.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/tags/editTag_p.java, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/config_p.java, htroot/api/feed.java, htroot/api/latency_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/api/trail_p.java, htroot/api/util/getpageinfo_p.java, htroot/api/util/ynetSearch.java, htroot/api/version.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/autoconfig.java, htroot/compare_yacy.java, htroot/cytag.java, htroot/imagetest.java, htroot/index.java, htroot/mediawiki_p.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java
Sun Oct 11 02:24:42 CEST 2009
by orbiter
refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6399 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/userDB.java, source/de/anomic/document/Document.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/ooxmlParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/server/ChunkedInputStream.java, source/de/anomic/http/server/ChunkedOutputStream.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/HeaderFramework.java, source/de/anomic/http/server/RequestHeader.java, source/de/anomic/http/server/ResponseContainer.java, source/de/anomic/http/server/ResponseHeader.java, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/http/server/servlets/transferURL.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/server/servletProperties.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/io/ByteCountInputStream.java, source/net/yacy/kelondro/io/ByteCountOutputStream.java, source/net/yacy/kelondro/util/MapTools.java, source/yacy.java
Sun Oct 11 02:12:19 CEST 2009
by orbiter
refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6398 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Bookmarks.java, htroot/Collage.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/CrawlResults.java, htroot/FeedReader_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/PeerLoadPicture.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/WatchCrawler_p.java, htroot/WebStructurePicture_p.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/api/util/getpageinfo_p.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/dbtest.java, source/de/anomic/content/DCEntry.java, source/de/anomic/content/dao/PhpBB3Dao.java, source/de/anomic/content/oai/PMHReader.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/ExternalIndexImporter.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/AbstractBlacklist.java, source/de/anomic/data/Blacklist.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/URLLicense.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/document/AbstractParser.java, source/de/anomic/document/Condenser.java, source/de/anomic/document/Document.java, source/de/anomic/document/Idiom.java, source/de/anomic/document/Parser.java, source/de/anomic/document/ParserException.java, source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/docParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/document/parser/html/ImageEntry.java, source/de/anomic/document/parser/html/ScraperInputStream.java, source/de/anomic/document/parser/html/TransformerWriter.java, source/de/anomic/document/parser/htmlParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/ooxmlParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/pptParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rssParser.java, source/de/anomic/document/parser/rtfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/swfParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/document/parser/vsdParser.java, source/de/anomic/document/parser/xlsParser.java, source/de/anomic/document/parser/zipParser.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/metadata/HeaderFramework.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/servlets/transferURL.java, source/de/anomic/net/ftpc.java, source/de/anomic/search/DocumentIndex.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/MetadataRepository.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ReferenceOrder.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Segment.java, source/de/anomic/search/Segments.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/search/blockrank/RCIEvaluation.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyUpdateLocation.java, source/de/anomic/ymage/WebStructureGraph.java, source/de/anomic/ymage/ymageOSM.java, source/net/yacy/kelondro/data/citation/CitationReferenceRow.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadata.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/navigation/NavigationReference.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceFactory.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/data/word/WordReference.java, source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/util/Domains.java, source/net/yacy/kelondro/util/ISO639.java, source/net/yacy/kelondro/util/Punycode.java, source/net/yacy/kelondro/workflow/AbstractBlockingThread.java, source/net/yacy/kelondro/workflow/AbstractBusyThread.java, source/net/yacy/kelondro/workflow/AbstractThread.java, source/net/yacy/kelondro/workflow/BlockingThread.java, source/net/yacy/kelondro/workflow/BusyThread.java, source/net/yacy/kelondro/workflow/InstantBlockingThread.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/kelondro/workflow/WorkflowJob.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java, source/net/yacy/kelondro/workflow/WorkflowThread.java, source/yacy.java, test/de/anomic/document/ParserTest.java, test/de/anomic/yacy/yacyURLTest.java
Sat Oct 10 03:14:19 CEST 2009
by orbiter
moved table and util classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6397 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImport_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/WatchCrawler_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/latency_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/cytag.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/yacy/message.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/dbtest.java, source/de/anomic/content/DCEntry.java, source/de/anomic/content/dao/ImportDump.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/ExternalIndexImporter.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/AbstractBlacklist.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java, source/de/anomic/document/Condenser.java, source/de/anomic/document/Document.java, source/de/anomic/document/Word.java, source/de/anomic/document/detector/odtDetector.java, source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/document/parser/html/ContentTransformer.java, source/de/anomic/document/parser/htmlParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/ooxmlParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rssParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/document/parser/xml/RSSReader.java, source/de/anomic/document/parser/xml/opensearchdescriptionReader.java, source/de/anomic/document/parser/zipParser.java, source/de/anomic/http/io/ChunkedInputStream.java, source/de/anomic/http/io/ChunkedOutputStream.java, source/de/anomic/http/metadata/HeaderFramework.java, source/de/anomic/http/metadata/RequestHeader.java, source/de/anomic/http/metadata/ResponseContainer.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/http/server/servlets/transferURL.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRDistribution.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/search/blockrank/RCIEvaluation.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverCachedFileOutputStream.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/server/serverSystem.java, source/de/anomic/tools/enumerateFiles.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFile.java, source/de/anomic/yacy/yacyAccessible.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/ymage/WebStructureGraph.java, source/de/anomic/ymage/ymageMatrix.java, source/migration.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapView.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/io/AbstractWriter.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/RecordStack.java, source/net/yacy/kelondro/table/Records.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/AttrSeq.java, source/net/yacy/kelondro/util/ByteArray.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/CompressedHashMap.java, source/net/yacy/kelondro/util/ConsoleInterface.java, source/net/yacy/kelondro/util/DateFormatter.java, source/net/yacy/kelondro/util/DiskSpace.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/Formatter.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/kelondro/util/NamePrefixThreadFactory.java, source/net/yacy/kelondro/util/ObjectSpace.java, source/net/yacy/kelondro/util/ScoreCluster.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/kelondro/util/SortStack.java, source/net/yacy/kelondro/util/SortStore.java, source/net/yacy/kelondro/util/XMLTables.java, source/net/yacy/kelondro/util/kelondroException.java, source/net/yacy/kelondro/util/kelondroOutOfLimitsException.java, source/yacy.java, test/de/anomic/kelondro/util/DateFormatterTest.java
Sat Oct 10 03:00:49 CEST 2009
by orbiter
moved io classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6396 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/table/Records.java, source/de/anomic/kelondro/table/Table.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/io/AbstractReader.java, source/net/yacy/kelondro/io/AbstractWriter.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/io/CachedFileWriter.java, source/net/yacy/kelondro/io/RandomAccessIO.java, source/net/yacy/kelondro/io/Reader.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/io/Writer.java
Sat Oct 10 02:43:25 CEST 2009
by orbiter
moved blob classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6395 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/http/client/Cache.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/yacy/yacySeedDB.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Gap.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapView.java, source/net/yacy/kelondro/blob/ObjectBuffer.java, source/net/yacy/kelondro/blob/Stack.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/yacy.java
Sat Oct 10 02:39:15 CEST 2009
by orbiter
accumulated classes from different packages into the new rwi package

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6394 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/api/timeline.java, htroot/yacy/search.java, source/de/anomic/crawler/ExternalIndexImporter.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/kelondro/blob/ArrayStack.java, source/de/anomic/kelondro/text/Metadata.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/kelondro/text/Segments.java, source/de/anomic/kelondro/text/citationPrototype/CitationReferenceRow.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReference.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceFactory.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceRow.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceVars.java, source/de/anomic/kelondro/text/referencePrototype/WordReference.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceFactory.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceVars.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/net/yacy/kelondro/rwi/AbstractBufferedIndex.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/rwi/BufferedIndex.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/Index.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/IndexReader.java, source/net/yacy/kelondro/rwi/Reference.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/ReferenceContainerOrder.java, source/net/yacy/kelondro/rwi/ReferenceFactory.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/net/yacy/kelondro/rwi/TermSearch.java, source/yacy.java, yacy.logging
Sat Oct 10 01:32:08 CEST 2009
by orbiter
moved kelondro index


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6393 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceMemory_p.java, htroot/Supporter.java, htroot/Surftips.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/document/Word.java, source/de/anomic/kelondro/blob/ArrayStack.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/table/RecordStack.java, source/de/anomic/kelondro/table/Records.java, source/de/anomic/kelondro/table/Relations.java, source/de/anomic/kelondro/table/SQLTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/Metadata.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/Reference.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceFactory.java, source/de/anomic/kelondro/text/citationPrototype/CitationReferenceRow.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReference.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceFactory.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceRow.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceVars.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceFactory.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceVars.java, source/de/anomic/kelondro/util/AttrSeq.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/net/yacy/kelondro/index/ARC.java, source/net/yacy/kelondro/index/BinSearch.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/index/ConcurrentARC.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/HandleSet.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/ObjectArray.java, source/net/yacy/kelondro/index/ObjectArrayCache.java, source/net/yacy/kelondro/index/ObjectIndex.java, source/net/yacy/kelondro/index/ObjectIndexCache.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/index/RowSetArray.java, source/net/yacy/kelondro/index/SimpleARC.java, source/yacy.java
Sat Oct 10 01:22:22 CEST 2009
by orbiter
moved kelondro order

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6392 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAccounts_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/SettingsAck_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/User.java, htroot/ViewProfile.java, htroot/WebStructurePicture_p.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacysearch.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/document/Condenser.java, source/de/anomic/document/Word.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/kelondro/blob/ArrayStack.java, source/de/anomic/kelondro/blob/BLOB.java, source/de/anomic/kelondro/blob/Compressor.java, source/de/anomic/kelondro/blob/Heap.java, source/de/anomic/kelondro/blob/HeapModifier.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/blob/MapDataMining.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/blob/ObjectBuffer.java, source/de/anomic/kelondro/blob/Stack.java, source/de/anomic/kelondro/index/BinSearch.java, source/de/anomic/kelondro/index/Cache.java, source/de/anomic/kelondro/index/HandleMap.java, source/de/anomic/kelondro/index/HandleSet.java, source/de/anomic/kelondro/index/IndexTest.java, source/de/anomic/kelondro/index/ObjectArrayCache.java, source/de/anomic/kelondro/index/ObjectIndex.java, source/de/anomic/kelondro/index/ObjectIndexCache.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/index/RowSetArray.java, source/de/anomic/kelondro/table/RecordStack.java, source/de/anomic/kelondro/table/Records.java, source/de/anomic/kelondro/table/Relations.java, source/de/anomic/kelondro/table/SQLTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/kelondro/text/AbstractBufferedIndex.java, source/de/anomic/kelondro/text/AbstractIndex.java, source/de/anomic/kelondro/text/BufferedIndex.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexReader.java, source/de/anomic/kelondro/text/Metadata.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceContainerOrder.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/kelondro/text/citationPrototype/CitationReferenceRow.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReference.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceVars.java, source/de/anomic/kelondro/util/AttrSeq.java, source/de/anomic/kelondro/util/ByteArray.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/search/blockrank/RCIEvaluation.java, source/de/anomic/tools/CryptoLib.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/ymage/WebStructureGraph.java, source/genpw.java, source/migration.java, source/net/yacy/kelondro/order/AbstractOrder.java, source/net/yacy/kelondro/order/Base64Order.java, source/net/yacy/kelondro/order/Bitfield.java, source/net/yacy/kelondro/order/ByteOrder.java, source/net/yacy/kelondro/order/CloneableIterator.java, source/net/yacy/kelondro/order/CloneableMapIterator.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/order/MicroDate.java, source/net/yacy/kelondro/order/NaturalOrder.java, source/net/yacy/kelondro/order/Order.java, source/net/yacy/kelondro/order/RotateIterator.java, source/net/yacy/kelondro/order/StackIterator.java, source/net/yacy/kelondro/order/StringOrder.java, source/yacy.java
Sat Oct 10 01:13:30 CEST 2009
by orbiter
start of a really extensive refactoring which will produce a hierarchical package structure with the domain yacy.net as package root
- moved here the logging classes as part of the new net.yacy.kelondro package

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6391 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/LogStatistics_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/message.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/de/anomic/crawler/AbstractImporter.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/ImporterManager.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/translator.java, source/de/anomic/document/AbstractParser.java, source/de/anomic/document/Parser.java, source/de/anomic/document/language/LanguageStatistics.java, source/de/anomic/document/language/LanguageStatisticsHolder.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/vsdParser.java, source/de/anomic/document/parser/xml/RSSReader.java, source/de/anomic/document/parser/xml/opensearchdescriptionReader.java, source/de/anomic/http/client/AcceptEverythingTrustManager.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/client/ConnectionInfo.java, source/de/anomic/http/metadata/ResponseHeader.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/kelondro/blob/ArrayStack.java, source/de/anomic/kelondro/blob/Compressor.java, source/de/anomic/kelondro/blob/HeapModifier.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/order/Digest.java, source/de/anomic/kelondro/table/RecordStack.java, source/de/anomic/kelondro/table/Records.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/kelondro/text/DocumentIndex.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/kelondro/text/Segments.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/kelondro/util/MemoryControl.java, source/de/anomic/net/UPnP.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRDistribution.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverAbstractThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/server/serverSwitchAbstractAction.java, source/de/anomic/server/serverSwitchAction.java, source/de/anomic/server/serverSystem.java, source/de/anomic/server/serverThread.java, source/de/anomic/tools/consoleInterface.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/tools/gzip.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/tools/tarTools.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/WebStructureGraph.java, source/de/anomic/ymage/ymageOSM.java, source/migration.java, source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java, source/net/yacy/kelondro/logging/ConsoleOutHandler.java, source/net/yacy/kelondro/logging/GuiHandler.java, source/net/yacy/kelondro/logging/Log.java, source/net/yacy/kelondro/logging/LogParser.java, source/net/yacy/kelondro/logging/LogalizerHandler.java, source/net/yacy/kelondro/logging/MiniLogFormatter.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java, source/yacy.java
Fri Oct 09 16:44:20 CEST 2009
by orbiter
* added index segments
This is a major change in the organization of indexes.
Please consider a back-up of your data before you run this update.
All existing index files will be moved and renamed to a new position.
With this change, it will be possible to maintain different indexes for different purposes and it will be possible to have a distinction between DHT-in and DHT-out specific indexes. Tenants may also have their own index, and it may be possible to have histories and back-ups of indexes. This is just the beginning, many servlets must be adopted after this change, but all functions that had been there should still work.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6389 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.init, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/Connections_p.java, htroot/CrawlResults.java, htroot/IndexCleaner_p.html, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/SettingsAck_p.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/WebStructurePicture_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/api/yacydoc.java, htroot/autoconfig.java, htroot/sharedBlacklist_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/message.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/de/anomic/content/oai/PMHReader.java, source/de/anomic/crawler/RobotsEntry.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/http/server/servlets/transferURL.java, source/de/anomic/kelondro/blob/ArrayStack.java, source/de/anomic/kelondro/blob/Compressor.java, source/de/anomic/kelondro/index/Column.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/text/DocumentIndex.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/kelondro/text/Segments.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/kelondro/util/ByteArray.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/yacy.java
Mon Oct 05 22:11:41 CEST 2009
by orbiter
- temporary integrated the transferURL servlet as static class instead as a class that is called using reflection to investigate the OOM problems in that class
- fixes for numerous other problems
- removed dead code
- resdesign of the strings-method, which produces now less memory overhead and may help to prevent OOMs
- another fix for the deadlock problem in SplitTable

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6373 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/CrawlProfileEditor_p.java, htroot/sharedBlacklist_p.java, htroot/yacy/transferRWI.java, source/de/anomic/content/oai/PMHReader.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/document/Parser.java, source/de/anomic/document/language/LanguageStatisticsHolder.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/servlets/transferURL.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/DocumentIndex.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/net/natLib.java, source/de/anomic/search/Switchboard.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java
Thu Oct 01 15:08:19 CEST 2009
by orbiter
- better logging for web cache access and fail reasons
- better Exception handling for web cache access
- distinction between access of web cache for proxy and crawler


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6367 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ViewFile.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/ymage/ymageOSM.java
Tue Sep 29 23:28:49 CEST 2009
by low012
*) blacklist cleaner supports usage of regular expressions now
*) refacored BlacklistCleaner_p.java for better readability
*) moved check of validity of patterns to the Balcklist implementation since patterns might be valid in one implementation, but not in another
*) added method to check validity to Blacklist interface
*) fixed some minor issues like typos or wrong whitespaces
*) set subversion properties for a whole bunch of files

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6359 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.html, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.html, htroot/Blacklist_p.java, htroot/IndexImportWikimedia_p.java, htroot/IndexImport_p.java, htroot/LogStatistics_p.java, htroot/PerformanceGraph.java, htroot/PerformanceSearch_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, source/de/anomic/data/AbstractBlacklist.java, source/de/anomic/data/Blacklist.java, source/de/anomic/data/Coordinates.java, source/de/anomic/data/DefaultBlacklist.java, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/data/LibraryProvider.java, source/de/anomic/data/Location.java, source/de/anomic/data/OpenGeoDB.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/URLLicense.java, source/de/anomic/data/diff.java
Sun Sep 27 00:07:40 CEST 2009
by low012
*) added subversion properties and edited a few comments

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6348 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigLiveSearch.java, htroot/ConfigNetwork_p.java, htroot/ConfigPHPBB3Search.java, htroot/ConfigParser.java, htroot/ConfigPortal.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/ConfigUpdate_p.java, htroot/ConfigWikiSearch.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/FeedReader_p.java, htroot/Help.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/compare_yacy.java, htroot/cytag.java, htroot/imagetest.java, htroot/mediawiki_p.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/ssitestservlet.java, htroot/test.java, htroot/yacyinteractive.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java
Wed Sep 23 23:26:14 CEST 2009
by low012
*) Removed base tag from errror page. This has been added by myself a long time ago as a workaround for some weird behavior of my router, but as it turns out, it does more bad than good in general: If HTTPS is used for communication with YaCy, entering a wrong passwort led to an errror page with a form which would send username and password unencrypted with the user possibly being unaware of this.
*) changed some comments, added some annotations, added SVN properties here and there

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6340 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/proxymsg/error.html, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/http/client/EasySSLProtocolSocketFactory.java, source/de/anomic/http/metadata/HeaderFramework.java, source/de/anomic/http/metadata/RequestHeader.java, source/de/anomic/http/metadata/ResponseContainer.java, source/de/anomic/http/metadata/ResponseHeader.java, source/de/anomic/http/server/AlternativeDomainNames.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/ProxyLogFormatter.java, source/de/anomic/http/server/RobotsTxtConfig.java, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/http/server/TemplateEngine.java
Mon Sep 14 23:17:42 CEST 2009
by orbiter
added a convenience class to add files into a YaCy index
to make this possible, the yacyURL must be able to process file:// urls, which has also been implemented
testing of the new class resulted in some bugfixes in other classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6313 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/document/parser/swfParser.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/DocumentIndex.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverDomains.java, source/de/anomic/tools/Punycode.java, source/de/anomic/yacy/yacyURL.java
Wed Sep 09 22:49:16 CEST 2009
by orbiter
refactoring of IO access classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6308 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Threaddump_p.java, source/de/anomic/kelondro/blob/Heap.java, source/de/anomic/kelondro/blob/HeapModifier.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/io/chunks/AbstractIOChunks.java, source/de/anomic/kelondro/io/chunks/IOChunksInterface.java, source/de/anomic/kelondro/io/chunks/RandomAccessIOChunks.java, source/de/anomic/kelondro/io/random/AbstractReader.java, source/de/anomic/kelondro/io/random/AbstractWriter.java, source/de/anomic/kelondro/io/random/CachedFileWriter.java, source/de/anomic/kelondro/io/random/FileWriter.java, source/de/anomic/kelondro/io/random/Reader.java, source/de/anomic/kelondro/io/random/Writer.java, source/de/anomic/kelondro/io/records/BufferedRecords.java, source/de/anomic/kelondro/io/records/Records.java, source/de/anomic/kelondro/table/Records.java, source/de/anomic/kelondro/table/Table.java
Mon Sep 07 22:30:57 CEST 2009
by orbiter
added configuration to enable ram table copy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6304 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/kelondro/table/Relations.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Sat Sep 05 22:41:21 CEST 2009
by low012
*) setting SVN keywords
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6297 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/search/IndexAbstracts.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/ResultFetcher.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/search/blockrank/CRDistribution.java, source/de/anomic/search/blockrank/RCIEvaluation.java
Sat Sep 05 22:04:23 CEST 2009
by low012
*) cleaned up code for better readability
*) added a few copyright notices
*) removed redundancy in constructors of ListToken

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6295 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/list/ListAccumulator.java, source/de/anomic/data/list/XMLBlacklistImporter.java, source/de/anomic/data/wiki/abstractWikiParser.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/AbstractToken.java, source/de/anomic/data/wiki/tokens/DefinitionListToken.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/data/wiki/tokens/ListToken.java, source/de/anomic/data/wiki/tokens/SimpleToken.java, source/de/anomic/data/wiki/tokens/TableToken.java, source/de/anomic/data/wiki/tokens/Token.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/data/wiki/wikiParser.java, source/de/anomic/data/wiki/wikiParserException.java
Sat Sep 05 01:33:47 CEST 2009
by orbiter
- corrected layout of map preview
- added caption to maps containing latitude and longitude information
- prevented that maps occur on second search page
- added location names to did-you-mean
- some refactoring of did-you-mean
- added equal and compareTo test to Coordinates class to make that work in set
- fixed utf-8 support for library files
- fixed a bug in images search icon view caption

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6294 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.html, source/de/anomic/data/Coordinates.java, source/de/anomic/data/DidYouMean.java, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/data/LibraryProvider.java, source/de/anomic/data/OpenGeoDB.java, source/de/anomic/ymage/ymageOSM.java
Wed Sep 02 15:41:56 CEST 2009
by orbiter
- inserted new library tests inside DidYouMean
- some redesign of DidYouMean that was necessary to follow
  a special rule how a library should be used:
  - the library provides words that start or end with a test
    word which may be possibly also an empty set of words
  - all words that the DidYouMean produced with the four
    production rules are used to generate a set of
    library-completed words
  - if this process results in any words from the library,
    only library-genrated words are taken
  - if the is no library-generated word at all, take the
    artifial generated word
  - all words that result from these rules are tested against
    the index
  - the result is ordered using a lightweight comparator that
    prefers short words
  - a not-so-much-io test against the index is beeing prepared
    next
- insered the library initialization into the switchboard

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6284 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/DidYouMean.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/Switchboard.java
Tue Sep 01 15:54:43 CEST 2009
by orbiter
added a library provider which holds libraries in static objects,
which can be used by any other classes to support their functions.
libraries are designed in such a way that users can create and
insert their own library files, but can also be imported from
other sources. As an example the "Korpusbasierte Wortgrundformliste
DeReWo des Institut für Deutsche Sprache" from
http://www.ids-mannheim.de has been integrated. This dictionary
is licensed to be used for all non-profit purposes. In case that
YaCy is used for commercial uses, this library must be removed.
The new library provilder reads the original source and translates
it into a simple word list to be used for the did-you-mean library
provider. More libraries may be provided in the future using
a download-servlet which puts files from the internet into the
<application-root>/dictionaries/ path.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6282 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: dictionaries/derewo-v-30000g-2007-12-31-0.1.pdf, dictionaries/derewo-v-30000g-2007-12-31-0.1.txt, dictionaries/readme.txt, source/de/anomic/data/DidYouMeanLibrary.java, source/de/anomic/data/LibraryProvider.java
Fri Aug 28 15:28:11 CEST 2009
by orbiter
some performance hacks (CPU only, not IO)
this will cause better computation speed for single- and multi-core;
there are enhancements that will speed up old and slow machines as well
as multi-core CPUs. Indexing of surrogates has been speed up
from 4000 PPM to over 20000 PPM on a simple dual core office computer.
Since the enhancements are mostly in core routines, the hack should also
speed up search performance.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6276 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceSearch_p.java, source/de/anomic/document/Condenser.java, source/de/anomic/document/Word.java, source/de/anomic/kelondro/index/ObjectIndexCache.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/util/SetTools.java, source/de/anomic/search/Switchboard.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/ymage/ProfilingGraph.java
Thu Aug 27 16:34:41 CEST 2009
by orbiter
refactoring of snippet cache

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6268 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/document/Condenser.java, source/de/anomic/document/Document.java, source/de/anomic/search/ImageCollector.java, source/de/anomic/search/MediaSnippet.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/SnippetFetcher.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/TextSnippet.java, source/de/anomic/yacy/yacyClient.java
Mon Aug 24 17:24:02 CEST 2009
by orbiter
refactoring of search process.
this is the beginning of some architecture changes that will hopefully bring some more stability, speed and transparency to the search process.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6260 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, htroot/Blacklist_p.java, htroot/IndexControlRWIs_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/search/IndexCollector.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/ResultEntry.java, source/de/anomic/search/SearchEvent.java, source/de/anomic/search/SearchEventCache.java, source/de/anomic/search/SnippetCache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/ymage/NetworkGraph.java, test/de/anomic/document/ParserTest.java
Wed Jul 29 21:43:11 CEST 2009
by orbiter
- patch to omit IndexOutOfBoundsException when a b64-encoded key appears not to be well-formed. In that case the key is still accepted but rated higher than other regular keys to create a virtual ordering between well-formed and ill-formed keys
- check routine at the beginning of the import of table keys that check that all imported keys are well-formed. All records that have a ill-formed key are deleted. This is a hack and is not tested since I don't have bad data here to test with. If the effect is seen in the wild, please report in the forum.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6245 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/client/Client.java, source/de/anomic/http/client/RemoteProxyConfig.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/order/ByteOrder.java, source/de/anomic/kelondro/order/NaturalOrder.java, source/de/anomic/kelondro/order/StringOrder.java, source/de/anomic/kelondro/table/Table.java
Sat Jul 25 23:38:57 CEST 2009
by orbiter
- fixes for some problems with the new crawling/caching strategies
- speed enhancements for the cache-only cache policy by using special no-delay rules in the balancer
- fixed some deadlock- and 100% CPU problems in the balancer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6243 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlStart_p.html, htroot/ViewImage.java, htroot/api/util/getpageinfo_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/document/parser/swfParser.java
Thu Jul 23 23:31:51 CEST 2009
by orbiter
redesign of access to the HTCache (now http.client.Cache):
- better control to the cache by using combined request-header and content access methods
- refactoring of many classes to comply to this new access method
- make shure that the cache is always written if something was loaded
- some redesign of the process how http response results are feeded into the new indexing queue
- introduction of a cache read policy:
 * never use the cache
 * use the cache if entry exist
 * use the cache if the proxy freshness rule confirmes
 * use only the cache and go never online
- added configuration options for the crawl profiles to use the new cache policies. There is not yet a input during crawl start to set the policy but this will be added in another step.
- set the default policies for the existing crawl profiles. If you want them to appear in your default profiles you must delete the crawl profiles database; othervise the policy is 'proxy freshness rule'
- enhanced some cache access methods in such a way that unnecessary retrievals are omitted (i.e. for size computation). That should reduce some IO but also a lot of CPU computation because sizes were computed after decompression of content after retrieval of the content from the disc.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6239 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/client/MultiOutputStream.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/kelondro/blob/Compressor.java, source/de/anomic/search/SnippetCache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/ymage/ymageOSM.java
Sun Jul 19 22:37:44 CEST 2009
by orbiter
refactoring:
- removed the plasma package. The name of that package came from a very early pre-version of YaCy, even before YaCy was named AnomicHTTPProxy. The Proxy project introduced search for cache contents using class files that had been developed during the plasma project. Information from 2002 about plasma can be found here:
http://web.archive.org/web/20020802110827/http://anomic.de/AnomicPlasma/index.html
We stil have one class that comes mostly unchanged from the plasma project, the Condenser class. But this is now part of the document package and all other classes in the plasma package can be assigned to other packages.
- cleaned up the http package: better structure of that class and clean isolation of server and client classes. The old HTCache becomes part of the client sub-package of http.
- because the plasmaSwitchboard is now part of the search package all servlets had to be touched to declare a different package source.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6232 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigLiveSearch.java, htroot/ConfigNetwork_p.java, htroot/ConfigPHPBB3Search.java, htroot/ConfigParser.java, htroot/ConfigPortal.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigUpdate_p.java, htroot/ConfigWikiSearch.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/FeedReader_p.java, htroot/Help.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImportWikimedia_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/Performance_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/Trails.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/tags/editTag_p.java, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/config_p.java, htroot/api/feed.java, htroot/api/latency_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/api/trail_p.java, htroot/api/util/getpageinfo_p.java, htroot/api/util/ynetSearch.java, htroot/api/version.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/autoconfig.java, htroot/compare_yacy.java, htroot/cytag.java, htroot/imagetest.java, htroot/index.java, htroot/mediawiki_p.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ExternalIndexImporter.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/listManager.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/document/Document.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/document/parser/html/ContentTransformer.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/http/client/AcceptEverythingSSLProtcolSocketFactory.java, source/de/anomic/http/client/AcceptEverythingTrustManager.java, source/de/anomic/http/client/Cache.java, source/de/anomic/http/client/Client.java, source/de/anomic/http/client/ClientGetMethod.java, source/de/anomic/http/client/ConnectionInfo.java, source/de/anomic/http/client/DefaultCharsetFilePart.java, source/de/anomic/http/client/DefaultCharsetStringPart.java, source/de/anomic/http/client/EasySSLProtocolSocketFactory.java, source/de/anomic/http/client/EasyX509TrustManager.java, source/de/anomic/http/client/MultiOutputStream.java, source/de/anomic/http/client/RemoteProxyConfig.java, source/de/anomic/http/io/BoundedSizeOutputStream.java, source/de/anomic/http/io/ByteCountInputStream.java, source/de/anomic/http/io/ByteCountOutputStream.java, source/de/anomic/http/io/ChunkedInputStream.java, source/de/anomic/http/io/ChunkedOutputStream.java, source/de/anomic/http/io/LimitExceededException.java, source/de/anomic/http/metadata/HeaderFramework.java, source/de/anomic/http/metadata/RequestHeader.java, source/de/anomic/http/metadata/ResponseContainer.java, source/de/anomic/http/metadata/ResponseHeader.java, source/de/anomic/http/server/AlternativeDomainNames.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/ProxyLogFormatter.java, source/de/anomic/http/server/RobotsTxtConfig.java, source/de/anomic/http/server/ServerSideIncludes.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/net/UPnP.java, source/de/anomic/net/natLib.java, source/de/anomic/search/IndexCollector.java, source/de/anomic/search/QueryEvent.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/SnippetCache.java, source/de/anomic/search/Switchboard.java, source/de/anomic/search/SwitchboardConstants.java, source/de/anomic/search/blockrank/CRDistribution.java, source/de/anomic/search/blockrank/CRProcess.java, source/de/anomic/search/blockrank/RCIEvaluation.java, source/de/anomic/server/serverDomains.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/servletProperties.java, source/de/anomic/tools/gzip.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/yacyAccessible.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/NetworkGraph.java, source/de/anomic/ymage/ProfilingGraph.java, source/de/anomic/ymage/WebStructureGraph.java, source/de/anomic/ymage/ymageOSM.java, source/migration.java, source/yacy.java
Fri Jul 17 15:59:21 CEST 2009
by orbiter
removed the indexing queue. This queue was superfluous since the introduction of the blocking queues last year, where documents are parsed, analysed and stored in the index with concurrency.
- The indexing queue was a historic data structure that was introduced at the very beginning at the project as a part of the switchboard organisation object structure. Without the indexing queue the switchboard queue becomes also superfluous. It has been removed as well.
- Removing the switchboard queue requires that all servlets are called without a opaque generic ('<?>'). That caused that all serlets had to be modified.
- Many servlets displayed the indexing queue or the size of that queue. In the past months the indexer was so fast that mostly the indexing queue appeared empty, so there was no use of it any more. Because the queue has been removed, the display in the servlets had also to be removed.
- The surrogate work task had been a part of the indexing queue control structure. Without the indexing queue the surrogates needed its own task management. That has been integrated here.
- Because the indexing queue had a special queue entry object and properties attached to this object, the propterties had to be moved to the queue entry object which is part of the new indexing queue withing the blocking queue, the Response Object. That object has now also the new properties of the removed indexing queue entry object.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6225 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigLiveSearch.java, htroot/ConfigNetwork_p.java, htroot/ConfigPHPBB3Search.java, htroot/ConfigParser.java, htroot/ConfigPortal.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigUpdate_p.java, htroot/ConfigWikiSearch.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/FeedReader_p.java, htroot/Help.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.html, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImportWikimedia_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/Performance_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Status_p.inc, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/Trails.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/delete_p.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/tags/editTag_p.java, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/config_p.java, htroot/api/feed.java, htroot/api/latency_p.java, htroot/api/queues_p.java, htroot/api/queues_p.xml, htroot/api/status_p.java, htroot/api/timeline.java, htroot/api/trail_p.java, htroot/api/util/getpageinfo_p.java, htroot/api/util/ynetSearch.java, htroot/api/version.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/autoconfig.java, htroot/compare_yacy.java, htroot/cytag.java, htroot/env/templates/submenuIndexCreate.template, htroot/imagetest.java, htroot/index.java, htroot/js/WatchCrawler.js, htroot/mediawiki_p.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/translator.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdRobotsTxtConfig.java, source/de/anomic/kelondro/blob/Heap.java, source/de/anomic/kelondro/blob/HeapModifier.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverPlainSwitch.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFile.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFtp.java, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacySeedUploader.java
Thu Jul 16 01:08:35 CEST 2009
by orbiter
- inserted request object into response object which carries this now instead generating new objects
- fixed a problem with the crawler introduced in SVN 6216

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6222 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexCreateWWWLocalQueue_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/io/BufferedEcoFS.java, source/de/anomic/kelondro/io/EcoFS.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/kelondro/text/ReferenceContainerOrder.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/search/SnippetCache.java, source/de/anomic/ymage/ymageOSM.java
Wed Jul 15 23:07:46 CEST 2009
by orbiter
-removed superfluous crawl cache
-refactoring of crawler classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6221 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/Network.java, htroot/QuickCrawlLink_p.java, htroot/Status.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/api/queues_p.java, htroot/api/util/getpageinfo_p.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/yacy/urls.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/LoaderDispatcher.java, source/de/anomic/crawler/retrieval/Request.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/rpmParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/text/Metadata.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/search/SnippetCache.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/ymage/ymageOSM.java
Wed Jul 15 18:22:56 CEST 2009
by lotus
suppress warnings for upnplib code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6219 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/net/sbbi/upnp/Discovery.java, source/net/sbbi/upnp/DiscoveryAdvertisement.java, source/net/sbbi/upnp/DiscoveryListener.java, source/net/sbbi/upnp/HttpResponse.java, source/net/sbbi/upnp/ServiceEventMessageParser.java, source/net/sbbi/upnp/ServicesEventing.java, source/net/sbbi/upnp/devices/UPNPDevice.java, source/net/sbbi/upnp/devices/UPNPRootDevice.java, source/net/sbbi/upnp/impls/InternetGatewayDevice.java, source/net/sbbi/upnp/messages/ActionMessage.java, source/net/sbbi/upnp/messages/ActionResponse.java, source/net/sbbi/upnp/services/ServiceAction.java, source/net/sbbi/upnp/services/ServiceStateVariable.java, source/net/sbbi/upnp/services/UPNPService.java
Wed Jul 15 18:13:24 CEST 2009
by lotus
* updated jxpath to latest v1.3
* added upnplib as source
	without packages:
	jmx
	remote
	samples

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6218 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, NOTICE, build.xml, lib/commons-jxpath-1.3.License, lib/commons-jxpath-1.3.jar, source/net/sbbi/upnp/Discovery.java, source/net/sbbi/upnp/DiscoveryAdvertisement.java, source/net/sbbi/upnp/DiscoveryEventHandler.java, source/net/sbbi/upnp/DiscoveryListener.java, source/net/sbbi/upnp/DiscoveryResultsHandler.java, source/net/sbbi/upnp/HttpResponse.java, source/net/sbbi/upnp/JXPathParser.java, source/net/sbbi/upnp/ServiceEventHandler.java, source/net/sbbi/upnp/ServiceEventMessageParser.java, source/net/sbbi/upnp/ServiceEventSubscription.java, source/net/sbbi/upnp/ServicesEventing.java, source/net/sbbi/upnp/devices/DeviceIcon.java, source/net/sbbi/upnp/devices/UPNPDevice.java, source/net/sbbi/upnp/devices/UPNPRootDevice.java, source/net/sbbi/upnp/impls/InternetGatewayDevice.java, source/net/sbbi/upnp/messages/ActionMessage.java, source/net/sbbi/upnp/messages/ActionMessageResponseParser.java, source/net/sbbi/upnp/messages/ActionResponse.java, source/net/sbbi/upnp/messages/StateVariableMessage.java, source/net/sbbi/upnp/messages/StateVariableResponse.java, source/net/sbbi/upnp/messages/StateVariableResponseParser.java, source/net/sbbi/upnp/messages/UPNPMessageFactory.java, source/net/sbbi/upnp/messages/UPNPResponseException.java, source/net/sbbi/upnp/services/ISO8601Date.java, source/net/sbbi/upnp/services/ServiceAction.java, source/net/sbbi/upnp/services/ServiceActionArgument.java, source/net/sbbi/upnp/services/ServiceStateVariable.java, source/net/sbbi/upnp/services/ServiceStateVariableTypes.java, source/net/sbbi/upnp/services/UPNPService.java, source/net/sbbi/upnp/version.properties
Wed Jul 15 16:15:51 CEST 2009
by orbiter
a set of small fixes to crawling behaviour

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6216 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexCreateIndexingQueue_p.java, htroot/Status.java, htroot/api/queues_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/document/Parser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyURL.java, source/yacy.java
Tue Jul 14 13:01:05 CEST 2009
by orbiter
enhanced document type recognition

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6209 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigParser.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/document/Idiom.java, source/de/anomic/document/Parser.java, source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/docParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/htmlParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/pptParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rpmParser.java, source/de/anomic/document/parser/rssParser.java, source/de/anomic/document/parser/rtfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/swfParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/document/parser/vsdParser.java, source/de/anomic/document/parser/xlsParser.java, source/de/anomic/document/parser/zipParser.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/search/SnippetCache.java
Mon Jul 13 12:19:10 CEST 2009
by orbiter
removed unused imports
removed unused libs from eclipse class path

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6201 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, htroot/Status.java, htroot/Steering.java, htroot/Threaddump_p.java, source/de/anomic/document/Parser.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Sat Jul 11 23:01:27 CEST 2009
by f1ori
enable warnings and fix most of it


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6196 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.xml, htroot/Blog.java, htroot/BlogComments.java, htroot/ConfigAccounts_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/CrawlResults.java, htroot/MessageSend_p.java, htroot/SearchEventPicture.java, htroot/Settings_p.java, htroot/User.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/http/httpDocument.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRequestHeader.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/index/HandleMap.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/citationPrototype/CitationReferenceRow.java, source/de/anomic/tools/DidYouMean.java, source/de/anomic/yacy/yacyNewsRecord.java
Sat Jul 11 19:03:22 CEST 2009
by f1ori
improvments to debianpackage:
* autoupdate completely disabled, display hint
* restart-button in interface works!

* moved all build-Variables to yacyBuildProperties
* fixed some warnings


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6195 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: addon/yacyInit.m4, build.xml, debian/changelog, debian/control, debian/rules, htroot/ConfigUpdate_p.html, htroot/ConfigUpdate_p.java, htroot/Status.java, htroot/Steering.java, htroot/Threaddump_p.java, htroot/User.java, htroot/ViewImage.java, htroot/ViewProfile.java, htroot/api/version.java, htroot/api/version.xml, htroot/opensearchdescription.java, htroot/www/welcome.java, htroot/yacy/hello.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyBuildProperties.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Fri Jul 10 18:15:33 CEST 2009
by orbiter
- enhanced logging and exception details for parsers
- removed inconsistencies in mime type declaration (one mime type should only appear once in all parsers)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6192 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/document/Parser.java, source/de/anomic/document/ParserException.java, source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/docParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/htmlParser.java, source/de/anomic/document/parser/mimeTypeParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/pptParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rtfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/plasma/plasmaSwitchboard.java
Fri Jul 10 16:22:17 CEST 2009
by orbiter
redesign of parser mime type detection and parser steering
There is now a mime-blacklist instead of a mime-whitelist

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6190 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/SettingsAck_p.java, htroot/Settings_p.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/document/AbstractParser.java, source/de/anomic/document/Classification.java, source/de/anomic/document/Document.java, source/de/anomic/document/Idiom.java, source/de/anomic/document/Parser.java, source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/docParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/htmlParser.java, source/de/anomic/document/parser/mimeTypeParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/pptParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rpmParser.java, source/de/anomic/document/parser/rssParser.java, source/de/anomic/document/parser/rtfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/swfParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/document/parser/vsdParser.java, source/de/anomic/document/parser/xlsParser.java, source/de/anomic/document/parser/zipParser.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/search/SnippetCache.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/yacyURL.java
Fri Jul 10 00:25:31 CEST 2009
by orbiter
refactoring of the ParserDispatcher and ParserConfig: resulted into Idiom, Parser and Classification classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6188 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/SettingsAck_p.java, htroot/Settings_p.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/document/AbstractParser.java, source/de/anomic/document/Classification.java, source/de/anomic/document/Condenser.java, source/de/anomic/document/Document.java, source/de/anomic/document/Idiom.java, source/de/anomic/document/Parser.java, source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/docParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/htmlParser.java, source/de/anomic/document/parser/mimeTypeParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/pptParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rpmParser.java, source/de/anomic/document/parser/rssParser.java, source/de/anomic/document/parser/rtfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/swfParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/document/parser/vsdParser.java, source/de/anomic/document/parser/xlsParser.java, source/de/anomic/document/parser/zipParser.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/order/NaturalOrder.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/search/SnippetCache.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/ymage/ymageChart.java
Thu Jul 09 22:56:30 CEST 2009
by orbiter
- some work to integrate the html parser the same way as the other parsers are integrated (not finished)
- added migration of code of settings pages (hmm.. does not work correctly yet, sorry)
- more refactoring
- removed more unused code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6187 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/SettingsAck_p.java, htroot/Settings_Parser.inc, htroot/Settings_p.java, source/de/anomic/document/AbstractParser.java, source/de/anomic/document/ParserDispatcher.java, source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/docParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/document/parser/htmlParser.java, source/de/anomic/document/parser/mimeTypeParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/pptParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rpmParser.java, source/de/anomic/document/parser/rssParser.java, source/de/anomic/document/parser/rtfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/swfParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/document/parser/vsdParser.java, source/de/anomic/document/parser/xlsParser.java, source/de/anomic/document/parser/zipParser.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/search/RankingProcess.java, source/migration.java
Thu Jul 09 22:13:11 CEST 2009
by low012
*) added changes which were lost
*) additional annotations
*) additional svn properties
*) _no_ functional changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6186 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/docParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/html/AbstractScraper.java, source/de/anomic/document/parser/html/AbstractTransformer.java, source/de/anomic/document/parser/html/CharacterCoding.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/document/parser/html/ContentTransformer.java, source/de/anomic/document/parser/html/ImageEntry.java, source/de/anomic/document/parser/html/Scraper.java, source/de/anomic/document/parser/html/ScraperInputStream.java, source/de/anomic/document/parser/html/ScraperListener.java, source/de/anomic/document/parser/html/Transformer.java, source/de/anomic/document/parser/html/TransformerWriter.java, source/de/anomic/document/parser/mimeTypeParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/pptParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rpmParser.java, source/de/anomic/document/parser/rssParser.java, source/de/anomic/document/parser/rtfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/swfParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/document/parser/xlsParser.java, source/de/anomic/document/parser/xml/RSSFeed.java, source/de/anomic/document/parser/xml/RSSReader.java, source/de/anomic/document/parser/xml/opensearchdescriptionReader.java, source/de/anomic/document/parser/zipParser.java
Thu Jul 09 00:14:57 CEST 2009
by orbiter
refactoring of search classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6184 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Blacklist_p.java, htroot/Bookmarks.java, htroot/IndexControlRWIs_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/api/timeline.java, htroot/index.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/search/ImageCollector.java, source/de/anomic/search/IndexCollector.java, source/de/anomic/search/QueryEvent.java, source/de/anomic/search/QueryParams.java, source/de/anomic/search/RankingProcess.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/search/SnippetCache.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java
Wed Jul 08 23:48:08 CEST 2009
by orbiter
refactoring of parsers and document processing

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6182 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: debian/changelog, htroot/Bookmarks.java, htroot/Connections_p.java, htroot/FeedReader_p.java, htroot/IndexControlRWIs_p.java, htroot/LogStatistics_p.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/feed.java, htroot/api/timeline.java, htroot/api/util/getpageinfo_p.java, htroot/cytag.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, source/de/anomic/content/DCEntry.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/wiki/wikiCode.java, source/de/anomic/document/AbstractParser.java, source/de/anomic/document/Condenser.java, source/de/anomic/document/Document.java, source/de/anomic/document/Parser.java, source/de/anomic/document/ParserConfig.java, source/de/anomic/document/ParserDispatcher.java, source/de/anomic/document/ParserException.java, source/de/anomic/document/Phrase.java, source/de/anomic/document/Word.java, source/de/anomic/document/detector/odtDetector.java, source/de/anomic/document/detector/rssDetector.java, source/de/anomic/document/language/Identificator.java, source/de/anomic/document/language/LanguageFilenameFilter.java, source/de/anomic/document/language/LanguageStatistics.java, source/de/anomic/document/language/LanguageStatisticsHolder.java, source/de/anomic/document/parser/bzipParser.java, source/de/anomic/document/parser/docParser.java, source/de/anomic/document/parser/gzipParser.java, source/de/anomic/document/parser/html/AbstractScraper.java, source/de/anomic/document/parser/html/AbstractTransformer.java, source/de/anomic/document/parser/html/CharacterCoding.java, source/de/anomic/document/parser/html/ContentScraper.java, source/de/anomic/document/parser/html/ContentTransformer.java, source/de/anomic/document/parser/html/ImageEntry.java, source/de/anomic/document/parser/html/Scraper.java, source/de/anomic/document/parser/html/ScraperInputStream.java, source/de/anomic/document/parser/html/ScraperListener.java, source/de/anomic/document/parser/html/Transformer.java, source/de/anomic/document/parser/html/TransformerWriter.java, source/de/anomic/document/parser/mimeTypeParser.java, source/de/anomic/document/parser/odtParser.java, source/de/anomic/document/parser/pdfParser.java, source/de/anomic/document/parser/pptParser.java, source/de/anomic/document/parser/psParser.java, source/de/anomic/document/parser/rpmParser.java, source/de/anomic/document/parser/rssParser.java, source/de/anomic/document/parser/rtfParser.java, source/de/anomic/document/parser/sevenzipParser.java, source/de/anomic/document/parser/swfParser.java, source/de/anomic/document/parser/tarParser.java, source/de/anomic/document/parser/vcfParser.java, source/de/anomic/document/parser/vsdParser.java, source/de/anomic/document/parser/xlsParser.java, source/de/anomic/document/parser/xml/RSSFeed.java, source/de/anomic/document/parser/xml/RSSReader.java, source/de/anomic/document/parser/xml/opensearchdescriptionReader.java, source/de/anomic/document/parser/zipParser.java, source/de/anomic/http/httpDocument.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/search/ImageCollector.java, source/de/anomic/search/IndexCollector.java, source/de/anomic/search/Query.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/urlRedirector.pl, source/de/anomic/server/urlRedirectord.java, source/de/anomic/tools/DidYouMean.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/logging/LogParser.java, source/de/anomic/yacy/logging/LogalizerHandler.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/ymage/ymageOSM.java, source/yacy.java
Wed Jul 08 22:53:08 CEST 2009
by low012
*) added several MIME types (derived from http://filext.com/), some of them might be rather uncommon
*) added an annotation forgotten in last commit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6180 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/plasma/parser/bzip/bzipParser.java, source/de/anomic/plasma/parser/doc/docParser.java, source/de/anomic/plasma/parser/gzip/gzipParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/rtf/rtfParser.java, source/de/anomic/plasma/parser/swf/swfParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/xls/xlsParser.java, source/de/anomic/plasma/parser/zip/zipParser.java
Wed Jul 08 21:20:03 CEST 2009
by low012
minor changes:
*) added annotations
*) set svn properties and added keywords to comments of parser classes
*) made a variable final to prevent (theoretical case of) change of object instance in synchronized block

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6179 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/plasma/parser/bzip/bzipParser.java, source/de/anomic/plasma/parser/doc/docParser.java, source/de/anomic/plasma/parser/gzip/gzipParser.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/plasma/parser/mimeType/rssDetector.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/rtf/rtfParser.java, source/de/anomic/plasma/parser/sevenzip/ByteArrayIInStream.java, source/de/anomic/plasma/parser/sevenzip/SZParserExtractCallback.java, source/de/anomic/plasma/parser/sevenzip/sevenzipParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/zip/zipParser.java
Mon Jul 06 17:21:43 CEST 2009
by orbiter
removed strange debugging strings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6177 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/kelondro/blob/Stack.java, source/de/anomic/kelondro/blob/Stacks.java, source/de/anomic/kelondro/index/HandleMap.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/order/Bitfield.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/tools/CryptoLib.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeed.java
Fri Jul 03 18:35:34 CEST 2009
by orbiter
- renamed Stack to RecordStack to avoid name confusion with new classes
- added new Stack class that implements a stack on BLOB files
- added new Stacks class that can be used for a set of Stacks (a 'Stack Database')
- added methods to other classes to support the new stacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6169 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/IndexingStack.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/Stack.java, source/de/anomic/kelondro/blob/Stacks.java, source/de/anomic/kelondro/index/HandleMap.java, source/de/anomic/kelondro/index/ObjectIndexCache.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/order/NaturalOrder.java, source/de/anomic/kelondro/table/RecordStack.java, source/de/anomic/yacy/yacyNewsQueue.java
Thu Jul 02 19:01:23 CEST 2009
by orbiter
all stack/heap files that had been stored in DATA/PLASMA are now stored in the network-specific QUEUES path
There is no migration. All crawls must be restarted.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6167 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/IndexImport_p.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/yacy.java
Thu Jul 02 14:33:47 CEST 2009
by f1ori
* fix double copy of libraries


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6164 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.xml, source/de/anomic/plasma/parser/bzip/build.xml, source/de/anomic/plasma/parser/doc/build.xml, source/de/anomic/plasma/parser/mimeType/build.xml, source/de/anomic/plasma/parser/odt/build.xml, source/de/anomic/plasma/parser/pdf/build.xml, source/de/anomic/plasma/parser/ppt/build.xml, source/de/anomic/plasma/parser/rpm/build.xml, source/de/anomic/plasma/parser/sevenzip/build.xml, source/de/anomic/plasma/parser/swf/build.xml, source/de/anomic/plasma/parser/tar/build.xml, source/de/anomic/plasma/parser/vcf/build.xml, source/de/anomic/plasma/parser/vsd/build.xml, source/de/anomic/plasma/parser/xls/build.xml, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.xml
Wed Jul 01 00:20:55 CEST 2009
by orbiter
code cleanup and code simplifcation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6161 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ViewFile.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/data/wiki/abstractWikiParser.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/http/httpdBoundedSizeOutputStream.java, source/de/anomic/kelondro/blob/Compressor.java, source/de/anomic/kelondro/blob/Heap.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/io/AbstractRandomAccess.java, source/de/anomic/kelondro/table/Records.java, source/de/anomic/kelondro/table/SQLTable.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/yacy/logging/ConsoleOutErrHandler.java, source/de/anomic/yacy/logging/ConsoleOutHandler.java, source/de/anomic/yacy/logging/MiniLogFormatter.java, source/de/anomic/yacy/logging/SimpleLogFormatter.java, source/de/anomic/yacy/yacySeedDB.java
Tue Jun 30 16:55:23 CEST 2009
by orbiter
moved all libraries from libx to lib
removed libx directory
all libraries are now in lib, instead the test libraries in libt which are not part of releases

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6157 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.xml, lib/FontBox-0.1.0-dev.jar, lib/FontBox.License, lib/J7Zip-modified.jar, lib/J7Zip.License, lib/PDFBox-0.7.3.License, lib/PDFBox-0.7.3.jar, lib/activation.jar, lib/bcmail-jdk14-139.jar, lib/bcprov-jdk14-139.jar, lib/bcprov.License, lib/commons-codec-1.3.License, lib/commons-codec-1.3.jar, lib/commons-discovery.jar, lib/gnumail.jar, lib/inetlib.jar, lib/jakarta-oro-2.0.7.jar, lib/jaxrpc.jar, lib/jmimemagic-0.1.0.jar, lib/jmimemagic-0.1.0.license, lib/jrpm-SNAPSHOT.jar, lib/jrpm-head.License, lib/jrpm-head.jar, lib/jsch-0.1.21.License, lib/jsch-0.1.21.jar, lib/log4j-1.2.9.jar, lib/odf_utils_05_11_29.jar, lib/poi-3.2-FINAL-20081019.jar, lib/poi-scratchpad-3.2-FINAL-20081019.jar, lib/readme_libx.txt, lib/tm-extractors-1.0.License, lib/tm-extractors-1.0.jar, lib/webcat-0.1-swf.License, lib/webcat-0.1-swf.jar, lib/wsdl4j.jar, source/de/anomic/plasma/parser/bzip/build.xml, source/de/anomic/plasma/parser/doc/build.xml, source/de/anomic/plasma/parser/mimeType/build.xml, source/de/anomic/plasma/parser/odt/build.xml, source/de/anomic/plasma/parser/pdf/build.xml, source/de/anomic/plasma/parser/ppt/build.xml, source/de/anomic/plasma/parser/rpm/build.xml, source/de/anomic/plasma/parser/sevenzip/build.xml, source/de/anomic/plasma/parser/swf/build.xml, source/de/anomic/plasma/parser/tar/build.xml, source/de/anomic/plasma/parser/vcf/build.xml, source/de/anomic/plasma/parser/vsd/build.xml, source/de/anomic/plasma/parser/xls/build.xml, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.xml
Tue Jun 30 15:25:46 CEST 2009
by orbiter
extending visibility of objects and methods to avoid synthetic accessor methods and increase performance

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6156 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .project, source/de/anomic/content/dao/PhpBB3Dao.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/diff.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/kelondro/blob/ArrayStack.java, source/de/anomic/kelondro/index/HandleMap.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/index/SimpleARC.java, source/de/anomic/kelondro/order/Digest.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/net/UPnP.java, source/de/anomic/server/serverCore.java, source/de/anomic/tools/DidYouMean.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/Tray.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/logging/Log.java
Tue Jun 30 15:01:35 CEST 2009
by orbiter
code cleanup: call of static methods directly to the class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6155 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blog.java, htroot/BlogComments.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/Connections_p.java, htroot/CrawlProfileEditor_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/QuickCrawlLink_p.java, htroot/Settings_p.java, htroot/User.java, htroot/ViewImage.java, htroot/Wiki.java, htroot/YaCySearchPluginFF.java, htroot/api/util/getpageinfo_p.java, htroot/autoconfig.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/httpClient.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Tue Jun 30 11:27:46 CEST 2009
by orbiter
simplification of the code: removed unused classes, methods and variables

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6154 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/dbtest.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/order/NaturalOrder.java, source/de/anomic/kelondro/table/Records.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceVars.java, source/de/anomic/plasma/parser/Condenser.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/search/Query.java, source/de/anomic/tools/crypt.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/ymage/ymageOSM.java
Tue Jun 30 00:22:35 CEST 2009
by orbiter
problems with code style

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6153 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .checkstyle, source/de/anomic/content/DCEntry.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/http/httpRequestHeader.java, source/de/anomic/kelondro/blob/ArrayStack.java, source/de/anomic/kelondro/index/HandleMap.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/net/UPnP.java, source/de/anomic/plasma/parser/Condenser.java, source/de/anomic/yacy/dht/Dispatcher.java
Sun Jun 28 23:37:37 CEST 2009
by orbiter
- After the removal of the Tree class some code simplifications are possible. This affects mostly the Records class, which can be refactored and the result of the refactoring results in a reduced number of classes.
- The EcoTable was renamed to Table.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6151 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/kelondro/table/Records.java, source/de/anomic/kelondro/table/Relations.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/table/Table.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/yacy/yacyNewsDB.java
Sun Jun 28 23:02:56 CEST 2009
by orbiter
completed migration of BLOBTree to BLOBHeaps:
- removed migration code
- removed BLOBTree
after the removal of the BLOBTree, a lot of dead code appeared:
- removed dead code that was needed for BLOBTree
Some more classes may have not much use any more after the removal of BLOBTree, but still have some component that are needed elsewhere. Additional Refactoring steps are needed to clean up dependencies and then more code may appear that is unused and can be removed as well.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6150 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/kelondro/index/Cache.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FullRecords.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/plasma/plasmaSwitchboard.java
Sat Jun 27 11:33:09 CEST 2009
by low012
*) updated Apache POI library to be able to parse Visio files
*) updated PPT and XLS parsers to use new Apache POI library
*) added new Visio (VSD) parser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6145 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: addon/jsmooth/yacy.jsmooth, defaults/yacy.init, libx/poi-3.2-FINAL-20081019.jar, libx/poi-scratchpad-3.2-FINAL-20081019.jar, source/de/anomic/plasma/parser/ppt/build.xml, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/vsd/build.xml, source/de/anomic/plasma/parser/vsd/vsdParser.java, source/de/anomic/plasma/parser/xls/build.xml, source/de/anomic/plasma/parser/xls/xlsParser.java
Fri Jun 26 11:11:02 CEST 2009
by apfelmaennchen
updated styles

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6141 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/themes/cupertino/images/ui-bg_diagonals-small_0_aaaaaa_40x40.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_diagonals-thick_15_444444_40x40.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_100_f0f0f0_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_50_99c2ff_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_55_fbf5d0_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_80_e6e6e6_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_highlight-hard_100_f9f9f9_1x100.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_highlight-soft_100_e7eef3_1x100.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_2694e8_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_72a7cf_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_888888_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/cupertino/jquery-ui.css, htroot/yacy/ui/css/themes/cupertino/ui.accordion.css, htroot/yacy/ui/css/themes/cupertino/ui.all.css, htroot/yacy/ui/css/themes/cupertino/ui.base.css, htroot/yacy/ui/css/themes/cupertino/ui.core.css, htroot/yacy/ui/css/themes/cupertino/ui.datepicker.css, htroot/yacy/ui/css/themes/cupertino/ui.dialog.css, htroot/yacy/ui/css/themes/cupertino/ui.progressbar.css, htroot/yacy/ui/css/themes/cupertino/ui.resizable.css, htroot/yacy/ui/css/themes/cupertino/ui.slider.css, htroot/yacy/ui/css/themes/cupertino/ui.tabs.css, htroot/yacy/ui/css/themes/cupertino/ui.theme.css, htroot/yacy/ui/css/themes/darkness/images/ui-bg_flat_30_cccccc_40x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_flat_50_5c5c5c_40x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_20_555555_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_40_0078a3_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_40_ffc73d_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_gloss-wave_25_333333_500x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_highlight-soft_80_eeeeee_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_inset-soft_25_000000_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_inset-soft_30_f58400_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_4b8e0b_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_a83300_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_cccccc_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/darkness/jquery-ui.css, htroot/yacy/ui/css/themes/darkness/ui.accordion.css, htroot/yacy/ui/css/themes/darkness/ui.all.css, htroot/yacy/ui/css/themes/darkness/ui.base.css, htroot/yacy/ui/css/themes/darkness/ui.core.css, htroot/yacy/ui/css/themes/darkness/ui.datepicker.css, htroot/yacy/ui/css/themes/darkness/ui.dialog.css, htroot/yacy/ui/css/themes/darkness/ui.progressbar.css, htroot/yacy/ui/css/themes/darkness/ui.resizable.css, htroot/yacy/ui/css/themes/darkness/ui.slider.css, htroot/yacy/ui/css/themes/darkness/ui.tabs.css, htroot/yacy/ui/css/themes/darkness/ui.theme.css, htroot/yacy/ui/css/themes/humanity/images/ui-bg_flat_75_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_100_f5f0e5_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_25_cb842e_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_70_ede4d4_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_100_f4f0ec_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_65_fee4bd_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_75_f5f5b5_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_inset-soft_100_f4f0ec_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_c47a23_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_cb672b_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_f08000_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_f35f07_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_ff7519_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/humanity/jquery-ui.css, htroot/yacy/ui/css/themes/humanity/ui.accordion.css, htroot/yacy/ui/css/themes/humanity/ui.all.css, htroot/yacy/ui/css/themes/humanity/ui.base.css, htroot/yacy/ui/css/themes/humanity/ui.core.css, htroot/yacy/ui/css/themes/humanity/ui.datepicker.css, htroot/yacy/ui/css/themes/humanity/ui.dialog.css, htroot/yacy/ui/css/themes/humanity/ui.progressbar.css, htroot/yacy/ui/css/themes/humanity/ui.resizable.css, htroot/yacy/ui/css/themes/humanity/ui.slider.css, htroot/yacy/ui/css/themes/humanity/ui.tabs.css, htroot/yacy/ui/css/themes/humanity/ui.theme.css, htroot/yacy/ui/css/themes/redmond/images/ui-bg_flat_0_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_flat_55_fbec88_40x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_75_d0e5f5_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_85_dfeffc_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_gloss-wave_55_5c9ccc_500x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_inset-hard_100_f5f8f9_1x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_inset-hard_100_fcfdfd_1x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_217bc0_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_469bdd_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_6da8d5_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_d8e7f3_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_f9bd01_256x240.png, htroot/yacy/ui/css/themes/redmond/jquery-ui.css, htroot/yacy/ui/css/themes/redmond/ui.accordion.css, htroot/yacy/ui/css/themes/redmond/ui.all.css, htroot/yacy/ui/css/themes/redmond/ui.base.css, htroot/yacy/ui/css/themes/redmond/ui.core.css, htroot/yacy/ui/css/themes/redmond/ui.datepicker.css, htroot/yacy/ui/css/themes/redmond/ui.dialog.css, htroot/yacy/ui/css/themes/redmond/ui.progressbar.css, htroot/yacy/ui/css/themes/redmond/ui.resizable.css, htroot/yacy/ui/css/themes/redmond/ui.slider.css, htroot/yacy/ui/css/themes/redmond/ui.tabs.css, htroot/yacy/ui/css/themes/redmond/ui.theme.css, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_flat_75_ffffff_40x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_65_ffffff_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_75_dadada_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_454545_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_888888_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/smoothness/jquery-ui.css, htroot/yacy/ui/css/themes/smoothness/ui.accordion.css, htroot/yacy/ui/css/themes/smoothness/ui.all.css, htroot/yacy/ui/css/themes/smoothness/ui.base.css, htroot/yacy/ui/css/themes/smoothness/ui.core.css, htroot/yacy/ui/css/themes/smoothness/ui.datepicker.css, htroot/yacy/ui/css/themes/smoothness/ui.dialog.css, htroot/yacy/ui/css/themes/smoothness/ui.progressbar.css, htroot/yacy/ui/css/themes/smoothness/ui.resizable.css, htroot/yacy/ui/css/themes/smoothness/ui.slider.css, htroot/yacy/ui/css/themes/smoothness/ui.tabs.css, htroot/yacy/ui/css/themes/smoothness/ui.theme.css, htroot/yacy/ui/css/themes/start/images/ui-bg_flat_55_999999_40x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_flat_75_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_glass_45_0078ae_1x400.png, htroot/yacy/ui/css/themes/start/images/ui-bg_glass_55_f8da4e_1x400.png, htroot/yacy/ui/css/themes/start/images/ui-bg_glass_75_79c9ec_1x400.png, htroot/yacy/ui/css/themes/start/images/ui-bg_gloss-wave_45_e14f1c_500x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_gloss-wave_50_6eac2c_500x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_gloss-wave_75_2191c0_500x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_inset-hard_100_fcfdfd_1x100.png, htroot/yacy/ui/css/themes/start/images/ui-icons_0078ae_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_056b93_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_d8e7f3_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_e0fdff_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_f5e175_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_f7a50d_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_fcd113_256x240.png, htroot/yacy/ui/css/themes/start/jquery-ui.css, htroot/yacy/ui/css/themes/start/ui.accordion.css, htroot/yacy/ui/css/themes/start/ui.all.css, htroot/yacy/ui/css/themes/start/ui.base.css, htroot/yacy/ui/css/themes/start/ui.core.css, htroot/yacy/ui/css/themes/start/ui.datepicker.css, htroot/yacy/ui/css/themes/start/ui.dialog.css, htroot/yacy/ui/css/themes/start/ui.progressbar.css, htroot/yacy/ui/css/themes/start/ui.resizable.css, htroot/yacy/ui/css/themes/start/ui.slider.css, htroot/yacy/ui/css/themes/start/ui.tabs.css, htroot/yacy/ui/css/themes/start/ui.theme.css
Wed Jun 17 11:58:15 CEST 2009
by orbiter
refactoring of blob package classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6088 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceMemory_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/userDB.java, source/de/anomic/kelondro/blob/ArrayStack.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/Compressor.java, source/de/anomic/kelondro/blob/Heap.java, source/de/anomic/kelondro/blob/HeapModifier.java, source/de/anomic/kelondro/blob/MapDataMining.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/index/Cache.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Tue Jun 16 23:45:40 CEST 2009
by orbiter
refactoring of search classes -- moved Ranking Profile to search package

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6086 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Ranking_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/search/Query.java, source/de/anomic/search/RankingProfile.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java
Mon Jun 15 23:19:54 CEST 2009
by orbiter
serialized all logging using concurrency:
high-performance search query situations as seen in yacy-metager integration showed deadlock situation caused by synchronization effects inside of sun.java code. It appears that the logger is not completely safe against deadlock situations in concurrent calls of the logger. One possible solution would be a outside-synchronization with 'synchronized' statements, but that would further apply blocking on all high-efficient methods that call the logger. It is much better to do a non-blocking hand-over of logging lines and work off log entries with a concurrent log writer. This also disconnects IO operations from logging, which can also cause IO operation when a log is written to a file. This commit not only moves the logger from kelondro to yacy.logging, it also inserts the concurrency methods to realize non-blocking logging.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6078 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/ProxyIndexingMonitor_p.java, htroot/ViewImage.java, htroot/yacy/crawlReceipt.java, htroot/yacy/message.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/de/anomic/crawler/AbstractImporter.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/ImporterManager.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/translator.java, source/de/anomic/http/AcceptEverythingTrustManager.java, source/de/anomic/http/HttpConnectionInfo.java, source/de/anomic/http/httpClient.java, source/de/anomic/http/httpResponseHeader.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/order/Digest.java, source/de/anomic/kelondro/table/AbstractRecords.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexWidthArray.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/kelondro/util/MemoryControl.java, source/de/anomic/language/identification/LanguageStatistics.java, source/de/anomic/language/identification/LanguageStatisticsHolder.java, source/de/anomic/net/UPnP.java, source/de/anomic/plasma/parser/AbstractParser.java, source/de/anomic/plasma/parser/Parser.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/sevenzip/SZParserExtractCallback.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverAbstractThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/server/serverSwitchAbstractAction.java, source/de/anomic/server/serverSwitchAction.java, source/de/anomic/server/serverSystem.java, source/de/anomic/server/serverThread.java, source/de/anomic/tools/DidYouMean.java, source/de/anomic/tools/consoleInterface.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/tools/gzip.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/tools/tarTools.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/logging/Log.java, source/de/anomic/yacy/logging/SimpleLogFormatter.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/ymageOSM.java, source/migration.java, source/yacy.java
Mon Jun 15 13:49:00 CEST 2009
by orbiter
refactoring: moved search query class to new search package

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6075 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/ContentIntegrationPHPBB3_p.html, htroot/IndexControlRWIs_p.java, htroot/Ranking_p.java, htroot/api/timeline.java, htroot/index.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/search/Query.java, source/de/anomic/yacy/yacySearch.java
Mon Jun 15 00:09:08 CEST 2009
by orbiter
a collection of
- small bug fixes
- better/more comments
- more asserts
- fixed synchronization
- test case enhancements
- code cleanup
- performance hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6073 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/BLOBHeap.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/index/IndexTest.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/util/ByteBuffer.java, source/de/anomic/plasma/plasmaSwitchboard.java
Sun Jun 14 01:03:40 CEST 2009
by orbiter
tried to insert a database dump import method to the phpBB3 import function. Reason: imports or large database dumps are cannot be handled with phpMyAdmin and this should be an easy way to the database dumps into a mySQL database where it can be exported again with the phpBB3 content integration adapter. Completion or removal of this function stub will follow before next main release.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6065 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ContentIntegrationPHPBB3_p.html, htroot/ContentIntegrationPHPBB3_p.java, source/de/anomic/content/dao/Dao.java, source/de/anomic/content/dao/DatabaseConnection.java, source/de/anomic/content/dao/ImportDump.java, source/de/anomic/content/dao/PhpBB3Dao.java
Sun Jun 14 00:59:54 CEST 2009
by orbiter
replaced rwi term counting method by one that computes the maximum of the blobs that contibute to the RWI. An addition of the blob sizes is wrong/incorrect and does not reflect the real size. Truncation the size operation to the maximum of all blobs is also incorrect, but not as wrong as the sum of all blob sizes wich double-counts many rwi entries.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6064 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/PerformanceGraph.java, htroot/api/queues_p.java, htroot/yacy/query.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/text/BufferedIndex.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaSwitchboard.java
Thu Jun 11 00:01:48 CEST 2009
by orbiter
added more word lists and a multi-access search test tool for high-performance query testing:
run searchtestmulti.sh; then 10 concurrent processes fire 1000 requests each to  the local peer.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6045 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: bin/searchtest.sh, bin/searchtest.words.aa, bin/searchtest.words.ab, bin/searchtest.words.ac, bin/searchtest.words.ad, bin/searchtest.words.ae, bin/searchtest.words.af, bin/searchtest.words.ag, bin/searchtest.words.ah, bin/searchtest.words.ai, bin/searchtest.words.aj, bin/searchtestmulti.sh
Tue Jun 09 01:30:12 CEST 2009
by orbiter
- completed the author navigation
- removed some unused variables

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6037 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java
Mon Jun 08 00:51:15 CEST 2009
by orbiter
added steering of navigator computation:
- by default the navigator computation if off for servlet yacysearch.html, but:
- the servlet is called by default with a option to switch navigator results on
this will prevent that metasearch users will get slow results that are caused by unnecessary computations

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6035 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/index.html, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java
Sun Jun 07 23:48:01 CEST 2009
by orbiter
- refactoring of IntegerHandleIndex and LongHandleIndex: both classes had been merged into the new HandleMap class, which handles (key<byte[]>,n-byte-long) pairs with arbitraty key and value length. This will be useful to get a memory-enhanced/minimized database table indexing.
- added a analysis method that counts bytes that could be saved in case the new HandleMap can be applied in the most efficient way. Look for the log messages beginning with "HeapReader saturation": in most cases we could save about 30% RAM!
- removed the old FlexTable database structure. It was not used any more.
- removed memory statistics in PerformanceMemory about flex tables and node caches (node caches were used by Tree Tables, which are also not used any more)
- add a stub for a steering of navigation functions. That should help to switch off naviagtion computation in cases where it is not demanded by a client

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6034 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, htroot/PerformanceMemory_p.xml, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/Column.java, source/de/anomic/kelondro/index/HandleMap.java, source/de/anomic/kelondro/index/IndexTest.java, source/de/anomic/kelondro/table/CachedRecords.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/plasma/plasmaSearchQuery.java
Sat Jun 06 20:09:58 CEST 2009
by orbiter
replaced tmp file extensions by prt

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6033 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/content/dao/PhpBB3Dao.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/Gap.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/kelondro/util/XMLTables.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/tools/mediawikiIndex.java
Sat Jun 06 03:56:31 CEST 2009
by orbiter
new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6025 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/net/ftpc.java, source/de/anomic/plasma/plasmaSwitchboard.java
Tue Jun 02 00:45:28 CEST 2009
by orbiter
enhanced navigation
- fixed too early computation of navigation
- moved navigation rendering to yacysearchtrailer
- added more asserts

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6006 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java
Mon Jun 01 23:49:18 CEST 2009
by orbiter
added more jquery themes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6005 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/themes/cupertino/images/theme.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_diagonals-small_0_aaaaaa_40x40.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_diagonals-thick_15_444444_40x40.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_100_f0f0f0_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_50_99c2ff_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_55_fbf5d0_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_80_e6e6e6_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_highlight-hard_100_f9f9f9_1x100.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_highlight-soft_100_e7eef3_1x100.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_2694e8_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_72a7cf_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_888888_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/cupertino/ui.accordion.css, htroot/yacy/ui/css/themes/cupertino/ui.all.css, htroot/yacy/ui/css/themes/cupertino/ui.base.css, htroot/yacy/ui/css/themes/cupertino/ui.core.css, htroot/yacy/ui/css/themes/cupertino/ui.datepicker.css, htroot/yacy/ui/css/themes/cupertino/ui.dialog.css, htroot/yacy/ui/css/themes/cupertino/ui.progressbar.css, htroot/yacy/ui/css/themes/cupertino/ui.resizable.css, htroot/yacy/ui/css/themes/cupertino/ui.slider.css, htroot/yacy/ui/css/themes/cupertino/ui.tabs.css, htroot/yacy/ui/css/themes/cupertino/ui.theme.css, htroot/yacy/ui/css/themes/darkness/images/theme.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_flat_30_cccccc_40x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_flat_50_5c5c5c_40x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_20_555555_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_40_0078a3_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_40_ffc73d_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_gloss-wave_25_333333_500x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_highlight-soft_80_eeeeee_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_inset-soft_25_000000_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_inset-soft_30_f58400_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_4b8e0b_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_a83300_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_cccccc_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/darkness/ui.accordion.css, htroot/yacy/ui/css/themes/darkness/ui.all.css, htroot/yacy/ui/css/themes/darkness/ui.base.css, htroot/yacy/ui/css/themes/darkness/ui.core.css, htroot/yacy/ui/css/themes/darkness/ui.datepicker.css, htroot/yacy/ui/css/themes/darkness/ui.dialog.css, htroot/yacy/ui/css/themes/darkness/ui.progressbar.css, htroot/yacy/ui/css/themes/darkness/ui.resizable.css, htroot/yacy/ui/css/themes/darkness/ui.slider.css, htroot/yacy/ui/css/themes/darkness/ui.tabs.css, htroot/yacy/ui/css/themes/darkness/ui.theme.css, htroot/yacy/ui/css/themes/humanity/images/theme.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_flat_75_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_100_f5f0e5_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_25_cb842e_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_70_ede4d4_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_100_f4f0ec_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_65_fee4bd_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_75_f5f5b5_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_inset-soft_100_f4f0ec_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_c47a23_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_cb672b_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_f08000_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_f35f07_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_ff7519_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/humanity/ui.accordion.css, htroot/yacy/ui/css/themes/humanity/ui.all.css, htroot/yacy/ui/css/themes/humanity/ui.base.css, htroot/yacy/ui/css/themes/humanity/ui.core.css, htroot/yacy/ui/css/themes/humanity/ui.datepicker.css, htroot/yacy/ui/css/themes/humanity/ui.dialog.css, htroot/yacy/ui/css/themes/humanity/ui.progressbar.css, htroot/yacy/ui/css/themes/humanity/ui.resizable.css, htroot/yacy/ui/css/themes/humanity/ui.slider.css, htroot/yacy/ui/css/themes/humanity/ui.tabs.css, htroot/yacy/ui/css/themes/humanity/ui.theme.css, htroot/yacy/ui/css/themes/redmond/images/theme.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_flat_0_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_flat_55_fbec88_40x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_75_d0e5f5_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_85_dfeffc_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_gloss-wave_55_5c9ccc_500x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_inset-hard_100_f5f8f9_1x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_inset-hard_100_fcfdfd_1x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_217bc0_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_469bdd_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_6da8d5_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_d8e7f3_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_f9bd01_256x240.png, htroot/yacy/ui/css/themes/redmond/ui.accordion.css, htroot/yacy/ui/css/themes/redmond/ui.all.css, htroot/yacy/ui/css/themes/redmond/ui.base.css, htroot/yacy/ui/css/themes/redmond/ui.core.css, htroot/yacy/ui/css/themes/redmond/ui.datepicker.css, htroot/yacy/ui/css/themes/redmond/ui.dialog.css, htroot/yacy/ui/css/themes/redmond/ui.progressbar.css, htroot/yacy/ui/css/themes/redmond/ui.resizable.css, htroot/yacy/ui/css/themes/redmond/ui.slider.css, htroot/yacy/ui/css/themes/redmond/ui.tabs.css, htroot/yacy/ui/css/themes/redmond/ui.theme.css, htroot/yacy/ui/css/themes/smoothness/images/theme.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_flat_75_ffffff_40x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_65_ffffff_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_75_dadada_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_454545_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_888888_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/smoothness/ui.accordion.css, htroot/yacy/ui/css/themes/smoothness/ui.all.css, htroot/yacy/ui/css/themes/smoothness/ui.base.css, htroot/yacy/ui/css/themes/smoothness/ui.core.css, htroot/yacy/ui/css/themes/smoothness/ui.datepicker.css, htroot/yacy/ui/css/themes/smoothness/ui.dialog.css, htroot/yacy/ui/css/themes/smoothness/ui.progressbar.css, htroot/yacy/ui/css/themes/smoothness/ui.resizable.css, htroot/yacy/ui/css/themes/smoothness/ui.slider.css, htroot/yacy/ui/css/themes/smoothness/ui.tabs.css, htroot/yacy/ui/css/themes/smoothness/ui.theme.css
Fri May 29 16:16:03 CEST 2009
by orbiter
- added new servlets to support search portal administrators for the integration of yacy search fields in their web pages
- moved some servlets from here to there..
- changed menu structure
- removed yacyui-portaltest.html which contained an example for the live search which is now integrated on all pages in yacy. The code snippet example from that page is integrated into the ConfigLiveSearch.html servlet


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5994 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.html, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.html, htroot/ConfigLiveSearch.html, htroot/ConfigLiveSearch.java, htroot/ConfigPHPBB3Search.html, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/ConfigProfile_p.html, htroot/ConfigRobotsTxt_p.html, htroot/ConfigWikiSearch.html, htroot/env/templates/footer.template, htroot/env/templates/header.template, htroot/env/templates/submenuConfig.template, htroot/env/templates/submenuCustomization.template, htroot/env/templates/submenuIntegration.template, source/de/anomic/yacy/yacySeed.java
Fri May 29 12:03:35 CEST 2009
by orbiter
more refactoring to make the segment object easier to use and to be prepared to integrate author navigation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5992 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/api/yacydoc.java, htroot/yacy/crawlReceipt.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/text/AbstractBufferedIndex.java, source/de/anomic/kelondro/text/AbstractIndex.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/plasma/parser/Condenser.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/yacy.java
Thu May 28 16:26:05 CEST 2009
by orbiter
refactoring of plasmaWordIndex:
divided that class into three parts:
- the peers object is now hosted by the plasmaSwitchboard
- the crawler elements are now in a new class, crawler.CrawlerSwitchboard
- the index elements are core of the new segment data structure, which is a bundle of different indexes for the full text and (in the future) navigation indexes and the metadata store. The new class is now in kelondro.text.Segment

The refactoring is inspired by the roadmap to create index segments, the option to host different indexes on one peer.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5990 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistImpExp_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigRobotsTxt_p.java, htroot/Connections_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/Wiki.java, htroot/YaCySearchPluginFF.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/api/util/ynetSearch.java, htroot/api/yacydoc.java, htroot/opensearchdescription.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/text/Segment.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacySearch.java, source/yacy.java
Wed May 27 17:04:04 CEST 2009
by orbiter
migration of all databases that use the deprecated BLOBTree format into the BLOBHeap format. Old databases are migrated automatically.
This removes the last very IO-intensive data structures which were still used for Wiki, Blog and Bookmarks. Old database files will still remain in the DATA subdirectory but can be deleted manually if no major bugs appear during migration. There is no need for any user action, all migration is done automatically.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5986 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/BLOBHeap.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/blob/MapDataMining.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/migration.java
Wed May 27 16:53:23 CEST 2009
by orbiter
added new submenu to crawler menu: import of phpbb3 forum postings from mysql
- yacy can import phpbb3 posts without crawling
- all data is written as surrogate
- indexed surrogate files can be re-used

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5985 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.init, htroot/ContentIntegrationPHPBB3_p.html, htroot/ContentIntegrationPHPBB3_p.java, htroot/env/templates/submenuIndexCreate.template, source/de/anomic/content/DCEntry.java, source/de/anomic/content/dao/Dao.java, source/de/anomic/content/dao/PhpBB3Dao.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/plasmaParserDocument.java
Tue May 26 09:44:22 CEST 2009
by orbiter
- refactoring: added new content package, which will contain connector classes for different types of data sources to import texts into the YaCy index
- refactoring: migrated data objects for the new connector classes
- added a DAO interface class to specify an abstract interface for database retrieval connector methods

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5977 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/FeedReader_p.java, htroot/api/feed.java, htroot/rct_p.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, source/de/anomic/content/DCEntry.java, source/de/anomic/content/RSSMessage.java, source/de/anomic/content/dao/Dao.java, source/de/anomic/content/file/SurrogateReader.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/xml/RSSFeed.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java
Tue May 26 00:27:34 CEST 2009
by orbiter
first attempt to add 'real' Navigation to yacy search results: host navigation
- after a search is started, it is analysed how many hits are in each site
- this can be done really efficient, because the navigation information is hidden in the url hash and can be computed very fast
- the search result shows a column on the right with the hosts and the hits per host
- after a click on a host the search is modified using the efficient site: - operator

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5976 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.html, htroot/yacysearch.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/yacy/yacySearch.java
Mon May 25 08:59:21 CEST 2009
by orbiter
increased default maximum file size for database files to 2GB
Other file sizes can now be configured with the attributes
filesize.max.win and filesize.max.other
the default maximum file size for non-windows OS is now 32GB

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5974 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, source/de/anomic/data/URLAnalysis.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/yacy.java
Fri May 22 01:53:01 CEST 2009
by orbiter
simplified code, removed one unused method in all implementing classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5972 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/kelondro/blob/Cache.java, source/de/anomic/kelondro/index/ObjectIndex.java, source/de/anomic/kelondro/index/ObjectIndexCache.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/index/RowSetArray.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/SQLTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Tree.java
Wed May 20 16:51:26 CEST 2009
by orbiter
The collection index will not be supported any more.
Existing indexes based on the old index collections must be migrated with YaCy 0.8
- removed index collection classes and all migration tools
- added a 'incell' reference collection feature in URL analysis


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5966 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/URLAnalysis.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/citationPrototype/CitationReferenceRow.java, source/de/anomic/plasma/parser/Condenser.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaWordIndex.java
Wed May 20 11:00:24 CEST 2009
by orbiter
- added prototype for navigation index
- refactoring of word index prototype
(no functional changes so far)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5965 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/text/Reference.java, source/de/anomic/kelondro/text/citationPrototype/CitationReferenceRow.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReference.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceFactory.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceRow.java, source/de/anomic/kelondro/text/navigationPrototype/NavigationReferenceVars.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceVars.java
Mon May 18 00:22:29 CEST 2009
by orbiter
replaced new navigation icons with same images but smaller resolution (16x16 instead of 128x128)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5961 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/env/grafics/navd0.gif, htroot/env/grafics/navd1.gif, htroot/env/grafics/navd10.gif, htroot/env/grafics/navd2.gif, htroot/env/grafics/navd3.gif, htroot/env/grafics/navd4.gif, htroot/env/grafics/navd5.gif, htroot/env/grafics/navd6.gif, htroot/env/grafics/navd7.gif, htroot/env/grafics/navd8.gif, htroot/env/grafics/navd9.gif, htroot/env/grafics/navdb.gif, htroot/env/grafics/navdl.gif, htroot/env/grafics/navdr.gif, htroot/env/grafics/navs0.gif, htroot/env/grafics/navs1.gif, htroot/env/grafics/navs10.gif, htroot/env/grafics/navs2.gif, htroot/env/grafics/navs3.gif, htroot/env/grafics/navs4.gif, htroot/env/grafics/navs5.gif, htroot/env/grafics/navs6.gif, htroot/env/grafics/navs7.gif, htroot/env/grafics/navs8.gif, htroot/env/grafics/navs9.gif, htroot/env/grafics/navsa.gif, htroot/env/grafics/navsb.gif, htroot/env/grafics/navsl.gif, htroot/env/grafics/navsr.gif
Fri May 15 01:11:10 CEST 2009
by orbiter
- added navigation icons for search result pages
- modified result page rendering to use new icons instead of numbers
- set different default values in yacy.init for higher indexing performance; removed pro-values
- modified WatchCrawler to accept 30000 PPM instead of only a maximum of 6000 PPM

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5952 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.init, htroot/ViewFile.java, htroot/WatchCrawler_p.html, htroot/WatchCrawler_p.java, htroot/env/grafics/navd0.gif, htroot/env/grafics/navd1.gif, htroot/env/grafics/navd10.gif, htroot/env/grafics/navd2.gif, htroot/env/grafics/navd3.gif, htroot/env/grafics/navd4.gif, htroot/env/grafics/navd5.gif, htroot/env/grafics/navd6.gif, htroot/env/grafics/navd7.gif, htroot/env/grafics/navd8.gif, htroot/env/grafics/navd9.gif, htroot/env/grafics/navdb.gif, htroot/env/grafics/navdl.gif, htroot/env/grafics/navdr.gif, htroot/env/grafics/navs0.gif, htroot/env/grafics/navs1.gif, htroot/env/grafics/navs10.gif, htroot/env/grafics/navs2.gif, htroot/env/grafics/navs3.gif, htroot/env/grafics/navs4.gif, htroot/env/grafics/navs5.gif, htroot/env/grafics/navs6.gif, htroot/env/grafics/navs7.gif, htroot/env/grafics/navs8.gif, htroot/env/grafics/navs9.gif, htroot/env/grafics/navsa.gif, htroot/env/grafics/navsb.gif, htroot/env/grafics/navsl.gif, htroot/env/grafics/navsr.gif, htroot/yacysearch.java
Tue May 05 08:31:35 CEST 2009
by orbiter
added a catch Exception to all thread to check if any of them silently dies without any other notification

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5922 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/blob/BLOBCompressor.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/tools/consoleInterface.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/xml/SurrogateReader.java, source/de/anomic/yacy/yacySearch.java
Sat May 02 14:12:22 CEST 2009
by f1ori
* splitt off yacyRelease from yacyVersion
* added some gui infos about signatures


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5916 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigUpdate_p.html, htroot/ConfigUpdate_p.java, htroot/Network.java, htroot/Steering.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyRelease.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Sat May 02 03:40:03 CEST 2009
by orbiter
- fixed possible bugs in Stack (may affect Crawler reset) and RandomAccess handling
- increased default memory size to 180MB
- fixed possible bug in http client reset (there was a deadlock)
- bug in BOBHeap marked, but not solved, cause is still unknown.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5912 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/Threaddump_p.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/http/httpClient.java, source/de/anomic/kelondro/blob/BLOBHeap.java, source/de/anomic/kelondro/io/CachedRandomAccess.java, source/de/anomic/kelondro/text/IODispatcher.java, startYACY.bat, startYACY.sh
Fri Apr 24 13:55:39 CEST 2009
by orbiter
attempt to fix a deadlock situation where the IODispatcher did not work.
I suspect the dispatcher thread has crashed and queues filled so no indexing process was able to write data.
This fix tries to heal the problem, but I am unsure if it helps. To get a better view of the problem, some more log outputs had been inserted.
Added also a new attribut indexer.threads to get a control over the number of default threads for the indexer (default is 1)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5866 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java
Thu Apr 23 00:39:12 CEST 2009
by orbiter
- removed useCell option: the indexCell data structure is now the default index structure; old collection data is still migrated
- added some debugging output to balancer to find a bug
- removed unused classes for index collection handling
- changed some default values for the process handling: more memory needed to prevent OOM

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5856 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/performance_dht.profile, defaults/yacy.init, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/IndexCollectionMigration.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/yacy.java
Wed Apr 22 00:12:19 CEST 2009
by orbiter
To translate a mediawiki dump into the YaCy surrogate format do the following:
- download a wikipedia dump, i.e. dewiki-20090311-pages-articles.xml.bz2
from http://download.wikimedia.org/dewiki/20090311/
- move dewiki-20090311-pages-articles.xml.bz2 to DATA/HTCACHE/
- start the conversion; open a command shell, move to the yacy home directory and execute
java -Xmx2000m -cp classes:lib/bzip2.jar de.anomic.tools.mediawikiIndex -convert DATA/HTCACHE/dewiki-20090311-pages-articles.xml.bz2 DATA/SURROGATES/in/ http://de.wikipedia.org/wiki/

this generates a series of files to DATA/SURROGATES/in

if YaCy is running (it may run concurrently), it fetches all new dumps in the surrogate-in directory. The export process is transaction-save, that means YaCy will not start reading a dump while the dump is not completely finished.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5851 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/httpdFileHandler.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/tools/mediawikiIndex.java
Tue Apr 21 16:23:04 CEST 2009
by orbiter
two patches for performance enhancements of the index handover process from documents to the index cache:
- one word prototype is generated for each document, that is re-used when a specific word is stored.
- the index cache uses now ByteArray objects to reference to the RWI instead of byte[]. This enhances access to the the map that stores the cache. To dump the cache to the FS, the content must be sorted, but sorting takes less time than maintenance of a sorted map during caching.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5849 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/index/IndexTest.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/IndexBuffer.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/plasma/plasmaWordIndex.java
Tue Apr 21 11:29:08 CEST 2009
by orbiter
more speed test cases: kelondro provides map functions that are more than 20% faster than standard java classes and use less than halve of the memory of java classes:
just start IndexTest (here with 1000000 test objects)

Performance test: comparing HashMap, TreeMap and kelondroRow
generated 1000000 test data entries 


STANDARD JAVA CLASS MAPS 

sorted map
time   for TreeMap<byte[]> generation: 2110
time   for TreeMap<byte[]> test: 2516, 0 bugs
memory for TreeMap<byte[]>: 29 MB

unsorted map
time   for HashMap<String> generation: 1157
time   for HashMap<String> test: 1516, 0 bugs
memory for HashMap<String>: 61 MB


KELONDRO-ENHANCED MAPS 

sorted map
time   for kelondroMap<byte[]> generation: 1781
time   for kelondroMap<byte[]> test: 2452, 0 bugs
memory for kelondroMap<byte[]>: 15 MB

unsorted map
time   for HashMap<ByteArray> generation: 828
time   for HashMap<ByteArray> test: 953, 0 bugs
memory for HashMap<ByteArray>: 9 MB

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5847 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/index/IndexTest.java, source/de/anomic/kelondro/util/ByteArray.java
Fri Apr 17 11:58:06 CEST 2009
by f1ori
* introduce signatures to autoupdate
  as long as there aren't publickeys for the updatelocations set,
  no signatures are checked
* wiki-article follows...


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5822 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.xml, htroot/ConfigUpdate_p.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/tools/CryptoLib.java, source/de/anomic/tools/SignatureOutputStream.java, source/de/anomic/yacy/yacyUpdateLocation.java, source/de/anomic/yacy/yacyVersion.java
Thu Apr 16 17:29:00 CEST 2009
by orbiter
full redesign of index access data model:
terms (words) are not any more retrieved by their word hash string, but by a byte[] containing the word hash.
this has strong advantages when RWIs are sorted in the ReferenceContainer Cache and compared with the sun.java TreeMap method, which needed getBytes() and new String() transformations before.
Many thousands of such conversions are now omitted every second, which increases the indexing speed by a factor of two.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5812 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/api/timeline.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/text/AbstractBufferedIndex.java, source/de/anomic/kelondro/text/AbstractIndex.java, source/de/anomic/kelondro/text/BufferedIndex.java, source/de/anomic/kelondro/text/BufferedIndexCollection.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexBuffer.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/IndexCollectionMigration.java, source/de/anomic/kelondro/text/IndexReader.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceContainerOrder.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/kelondro/util/SetTools.java, source/de/anomic/net/ftpc.java, source/de/anomic/plasma/parser/Word.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverObjects.java, source/de/anomic/xml/SurrogateReader.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/dht/VerticalWordPartitionScheme.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Thu Apr 16 10:01:38 CEST 2009
by orbiter
- added default profile for surrogate indexing
- integrated surrogate indexing into indexing queue process

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5810 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/CrawlProfileEditor_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/WatchWebStructure_p.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/xml/SurrogateReader.java
Wed Apr 15 08:34:27 CEST 2009
by orbiter
- full abstraction of index content type: the kelondro full text index may now also contain indexes about other content than text, i.e. navigation indexes or reverse linking indexes.
- during index joins all word positions are maintained: better ranking for word distance possible; exact phrase match can be implemented soundly


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5804 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/IndexControlRWIs_p.java, htroot/api/timeline.java, htroot/yacy/search.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/text/AbstractBufferedIndex.java, source/de/anomic/kelondro/text/AbstractIndex.java, source/de/anomic/kelondro/text/AbstractReference.java, source/de/anomic/kelondro/text/BufferedIndex.java, source/de/anomic/kelondro/text/BufferedIndexCollection.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexBuffer.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/IndexCollectionMigration.java, source/de/anomic/kelondro/text/IndexReader.java, source/de/anomic/kelondro/text/Reference.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceContainerOrder.java, source/de/anomic/kelondro/text/ReferenceFactory.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/referencePrototype/WordReference.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceFactory.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceVars.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/yacy.java
Wed Apr 08 17:28:45 CEST 2009
by orbiter
refactoring of wiki parser.
This was done to prepare the wiki parser as parser for wikipedia dumps, which will be used for performance test (to omit crawling)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5785 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.xml, htroot/ViewProfile.java, htroot/Wiki.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/wiki/abstractWikiParser.java, source/de/anomic/data/wiki/wikiBoard.java, source/de/anomic/data/wiki/wikiCode.java, source/de/anomic/data/wiki/wikiParser.java, source/de/anomic/plasma/parser/bzip/bzipParser.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/server/serverObjects.java, source/de/anomic/tools/mediawikiIndex.java
Tue Apr 07 11:34:41 CEST 2009
by orbiter
- refactoring
- a little bit more abstraction
- new interfaces for index abstraction

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5783 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacy/transferRWI.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/Reference.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceFactory.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/kelondro/text/referencePrototype/CitationReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReference.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceVars.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/yacy.java
Fri Apr 03 15:23:45 CEST 2009
by orbiter
refactoring: better abstraction of reference and metadata prototypes.
This is a preparation to introduce other index tables as used now only for reverse text indexes. Next application of the reverse index is a citation index.
Moved to version 0.74

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5777 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/BlacklistCleaner_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/api/blacklists_p.java, htroot/api/timeline.java, htroot/api/yacydoc.java, htroot/sharedBlacklist_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/data/AbstractBlacklist.java, source/de/anomic/data/Blacklist.java, source/de/anomic/data/DefaultBlacklist.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLAnalysis.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/listManager.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/text/BufferedIndexCollection.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexBuffer.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/IndexCollectionMigration.java, source/de/anomic/kelondro/text/Metadata.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/Reference.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceContainerOrder.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/metadataPrototype/URLMetadataRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceRow.java, source/de/anomic/kelondro/text/referencePrototype/WordReferenceVars.java, source/de/anomic/plasma/parser/Condenser.java, source/de/anomic/plasma/parser/Document.java, source/de/anomic/plasma/parser/Phrase.java, source/de/anomic/plasma/parser/Word.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/ymage/ymageOSM.java, source/yacy.java
Thu Apr 02 17:08:56 CEST 2009
by orbiter
a different naming scheme for BLOBArray files. This may be necessary if blobs are written more often than once in a second.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5771 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceGraph.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/index/ObjectIndex.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/util/DateFormatter.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaWordIndex.java
Thu Apr 02 15:26:47 CEST 2009
by orbiter
- added fast site-operator
- refactoring merge into BLOBArray

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5770 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/api/webstructure.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyURL.java
Wed Apr 01 16:53:23 CEST 2009
by orbiter
added web structure analysis for a special domain that can be requested from the api.

Example:
http://localhost:8080/api/webstructure.xml?about=www.yacy.net
returns a xml with the following content:

<?xml version="1.0"?>
<webstructure>
<domains reference="reverse" count="1" maxref="300">
<domain host="www.yacy.net" id="FXg39Q" date="20090401">
  <citation host="java.sun.com" id="o-R3yY" count="1" />
  <citation host="yacy-suche.de" id="-KCLaB" count="1" />
  <citation host="suma-ev.de" id="VRAHIA" count="1" />
  <citation host="www.kit.edu" id="EMaLDQ" count="1" />
  <citation host="yacy.net" id="Fh1hyQ" count="1" />
  <citation host="www.fzk.de" id="V2Kl-A" count="1" />
  <citation host="en.wikipedia.org" id="rwtdfR" count="3" />
  <citation host="vimeo.com" id="MmdQDY" count="3" />
  <citation host="liebel.fzk.de" id="sX4ozA" count="6" />
</domain>
</domains>
</webstructure>


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5766 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/WebStructurePicture_p.java, htroot/api/webstructure.java, source/de/anomic/plasma/plasmaWebStructure.java
Wed Apr 01 14:39:11 CEST 2009
by orbiter
- added writing of temporary file names and renaming to final file name when index dump/merge are done. Interrupted merges can be cleaned up.
- added clean-up of unfinished merges and unused idx/gap files
- enhanced merge file selection method

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5764 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/Gap.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java
Tue Mar 31 18:49:02 CEST 2009
by orbiter
- better selection of files to be merged
- fix for getChannel().close(), which works on windows but not on macs and linux

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5761 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/io/CachedRandomAccess.java, source/de/anomic/kelondro/order/Digest.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollectionMigration.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/tools/mediawikiIndex.java
Tue Mar 31 14:42:12 CEST 2009
by orbiter
more care about open file handles. Now files also close on windows and can be deleted afterwards.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5760 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/kelondro/blob/Gap.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/io/CachedRandomAccess.java, source/de/anomic/kelondro/order/Digest.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/tools/mediawikiIndex.java
Mon Mar 30 17:31:25 CEST 2009
by orbiter
- added migration class to go from index collections to the index cell data structure.
- added better control over file deletion, because this sometimes fails, especially on windows

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5756 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigUpdate_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/userDB.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/BLOBHeap.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/io/CachedRandomAccess.java, source/de/anomic/kelondro/io/EcoFS.java, source/de/anomic/kelondro/table/AbstractRecords.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FixedWidthArray.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/FlexWidthArray.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/kelondro/text/BufferedIndexCollection.java, source/de/anomic/kelondro/text/IODispatcher.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/IndexCollectionMigration.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/kelondro/util/XMLTables.java, source/de/anomic/plasma/parser/bzip/bzipParser.java, source/de/anomic/plasma/parser/gzip/gzipParser.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverSystem.java, source/de/anomic/tools/enumerateFiles.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java
Fri Mar 27 07:04:39 CET 2009
by apfelmaennchen
RichClient:
- renamed base theme to start theme
- reoved all but start theme
- additional themes can be downloaded from http://jquery-ui.googlecode.com/files/jquery-ui-themes-1.7.zip
- or a custom theme can be generated at http://jqueryui.com/themeroller/
- themes are installed into DATA/LOCALE/htroot/yacy/ui/css/themes
- update for RichClient theme selection will follow soon

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5743 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/themes/start/images/ui-bg_flat_55_999999_40x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_flat_75_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_glass_45_0078ae_1x400.png, htroot/yacy/ui/css/themes/start/images/ui-bg_glass_55_f8da4e_1x400.png, htroot/yacy/ui/css/themes/start/images/ui-bg_glass_75_79c9ec_1x400.png, htroot/yacy/ui/css/themes/start/images/ui-bg_gloss-wave_45_e14f1c_500x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_gloss-wave_50_6eac2c_500x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_gloss-wave_75_2191c0_500x100.png, htroot/yacy/ui/css/themes/start/images/ui-bg_inset-hard_100_fcfdfd_1x100.png, htroot/yacy/ui/css/themes/start/images/ui-icons_0078ae_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_056b93_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_d8e7f3_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_e0fdff_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_f5e175_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_f7a50d_256x240.png, htroot/yacy/ui/css/themes/start/images/ui-icons_fcd113_256x240.png, htroot/yacy/ui/css/themes/start/jquery-ui.css, htroot/yacy/ui/css/themes/start/ui.accordion.css, htroot/yacy/ui/css/themes/start/ui.all.css, htroot/yacy/ui/css/themes/start/ui.base.css, htroot/yacy/ui/css/themes/start/ui.core.css, htroot/yacy/ui/css/themes/start/ui.datepicker.css, htroot/yacy/ui/css/themes/start/ui.dialog.css, htroot/yacy/ui/css/themes/start/ui.progressbar.css, htroot/yacy/ui/css/themes/start/ui.resizable.css, htroot/yacy/ui/css/themes/start/ui.slider.css, htroot/yacy/ui/css/themes/start/ui.tabs.css, htroot/yacy/ui/css/themes/start/ui.theme.css, htroot/yacy/ui/index.html, htroot/yacy/ui/yacyui-portaltest.html, htroot/yacy/ui/yacyui-widget.html
Fri Mar 20 16:44:59 CET 2009
by borg-0300
svn attributes added

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5736 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/text/Document.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexBuffer.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexReader.java, source/de/anomic/kelondro/text/MetadataRowContainer.java, source/de/anomic/kelondro/text/Phrase.java, source/de/anomic/kelondro/text/Reference.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceContainerMerger.java, source/de/anomic/kelondro/text/ReferenceContainerOrder.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/ReferenceRow.java, source/de/anomic/kelondro/text/ReferenceVars.java, source/de/anomic/kelondro/text/URLMetadata.java, source/de/anomic/kelondro/text/Word.java
Fri Mar 20 11:21:23 CET 2009
by orbiter
- added Latency control to the crawler:
because of the strongly enhanced indexing speed when using the new IndexCell RWI data structures (> 2000PPM on my notebook), it is now necessary to control the crawling speed depending on the response time of the target server (which is also YaCy in case of some intranet indexing use cases).
The latency factor in crawl delay times is derived from the time that a target hosts takes to answer on http requests. For internet domains, the crawl delay is a minimum of twice the response time, in intranet cases the delay time is now a halve of the response time.

- added API to monitor the latency times of the crawler:
a new api at /api/latency_p.xml returns the current response times of domains, the time when the domain was accessed by the crawler the last time and many more attributes.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5733 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/api/latency_p.java, htroot/api/latency_p.xml, htroot/api/webstructure.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/Latency.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java
Wed Mar 18 21:21:19 CET 2009
by orbiter
- enhanced balancer: auto-solving of waiting-deadlocks
- removed deprecated cache-init size value
- more debug lines for IndexCell cache dump merge

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5728 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, htroot/PerformanceQueues_p.xml, source/de/anomic/crawler/Balancer.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java
Wed Mar 18 17:14:31 CET 2009
by orbiter
added a merge operation for IndexCell data structures

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5727 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/httpClient.java, source/de/anomic/kelondro/blob/BLOB.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/BLOBCompressor.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/text/IndexBuffer.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java
Tue Mar 17 14:03:27 CET 2009
by orbiter
Integration of the new index data structure IndexCell
This is the start of a testing phase for IndexCell data structure which will replace
the collections and caching strategy. IndexCall creation and maintenance is fast, has
no caching overhead, very low IO load and is the basis for the next data structure,
index segments.

IndexCell files are stored at DATA/<network>/TEXT/RICELL
With this commit still the old data structures are used, until a flag in yacy.conf is set.
To switch to the new data structure, set
useCell = true
in yacy.conf. Then you will have no access any more to TEXT/RICACHE and TEXT/RICOLLECTION

This code is still bleeding-edge development. Please do not use the new data structure for
production now. Future versions may have changed data types, or other storage locations.
The next main release will have a migration feature for old data structures.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5724 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.init, source/de/anomic/kelondro/text/BufferedIndex.java, source/de/anomic/kelondro/text/BufferedIndexCollection.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/yacy.java
Mon Mar 16 17:24:53 CET 2009
by orbiter
more refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5722 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/text/AbstractBufferedIndex.java, source/de/anomic/kelondro/text/AbstractIndex.java, source/de/anomic/kelondro/text/BufferedIndex.java, source/de/anomic/kelondro/text/BufferedIndexCollection.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexBuffer.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/IndexReader.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/yacy.java
Mon Mar 16 09:32:28 CET 2009
by orbiter
more refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5719 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/transferRWI.java, source/de/anomic/kelondro/text/CachedIndexCollection.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexCache.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/IndexPackage.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceRow.java, source/de/anomic/plasma/plasmaWordIndex.java
Mon Mar 16 01:18:37 CET 2009
by orbiter
refactoring to integrate indexCell data structures

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5718 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/PerformanceGraph.java, htroot/yacy/query.java, htroot/yacysearch.java, source/de/anomic/kelondro/text/AbstractIndex.java, source/de/anomic/kelondro/text/CachedIndexCollection.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexCache.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/IndexPackage.java, source/de/anomic/kelondro/text/IndexReader.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java
Fri Mar 13 17:52:31 CET 2009
by orbiter
removed option to use different primary keys in kelondro tables
this option was never used and there is also no use to set other columns but the first as the primary key. as a result, access methods to the key do not need to compute key positions, and they work faster.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5711 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Supporter.java, htroot/Surftips.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/Cache.java, source/de/anomic/kelondro/index/HandleSet.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/index/ObjectArrayCache.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/table/AbstractRecords.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FixedWidthArray.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/FlexWidthArray.java, source/de/anomic/kelondro/table/Hashtable.java, source/de/anomic/kelondro/table/Relations.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/MetadataRowContainer.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceRow.java, source/de/anomic/kelondro/util/AttrSeq.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java
Fri Mar 13 15:56:25 CET 2009
by orbiter
refactoring of plasmaWordIndex: less methods in the class, separated the index to CachedIndexCollection

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5710 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/timeline.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/text/CachedIndexCollection.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java, source/yacy.java
Fri Mar 13 11:34:51 CET 2009
by orbiter
refactoring of wordIndex class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5709 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistImpExp_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigRobotsTxt_p.java, htroot/Connections_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/Wiki.java, htroot/YaCySearchPluginFF.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/add_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/util/ynetSearch.java, htroot/api/yacydoc.java, htroot/opensearchdescription.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/wiki/abstractWikiParser.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacySearch.java
Fri Mar 13 11:07:04 CET 2009
by orbiter
more performance hacks: better default values for scaling, less memory usage

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5708 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/URLAnalysis.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/HandleSet.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/index/ObjectIndexCache.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/IndexCollection.java
Fri Mar 13 01:13:47 CET 2009
by orbiter
replaced old PPM computation with a better one that simply sums up events that had been stored in the profiling table.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5706 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchtrailer.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/index/ObjectIndexDaemon.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverProfiling.java
Fri Mar 13 00:05:18 CET 2009
by orbiter
added new class RowSetArray which arranges RowSet objects like Elements in a hashtable, but still provides the functionality of sorted enumeration. The new class is now integrated into the ObjectIndexCache, which is the core class to provide index functions to all database files. The new index access is about twice as fast as before. This has strong speed enhancement effects on all parts of YaCy.

The speed of the kelondro indexing class ObjectIndexCache can be compared with Javas standard TreeMap with the main method in IntegerHandleIndex. The result is, that the kelondro indexing needs only 1/5 of the memory that TreeMap uses! In exchange, the kelondro classes are slower than TreeMap, about four (!) times slower. However, this is not so bad because the better use of the memory is a strong advantage and makes it possible that YaCy can maintain such a large number of document (> 50 million) in one peer.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5705 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/index/ObjectIndexCache.java, source/de/anomic/kelondro/index/RowSetArray.java, source/de/anomic/kelondro/order/MergeIterator.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/MetadataRepository.java
Wed Mar 11 21:23:19 CET 2009
by orbiter
performance hack to ObjectIndex put() method:
Java standard classes provide a Map Interface, that has a put() method that returns the object that was replaced by the object that was the argument of the put call. The kelondro ObjectIndex defined a put method in the same way, that means it also returned the previous value of the Entry object before the put call. However, this value was not used by the calling code in the most cases. Omitting a return of the previous value would cause some performance benefit. This change implements a put method that does not return the previous value to reflect the common use. Omitting the return of previous values will cause some benefit in performance. The functionality to get the previous value is still maintained, and provided with a new 'replace' method. 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5700 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/blob/Cache.java, source/de/anomic/kelondro/index/HandleSet.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/index/ObjectArrayCache.java, source/de/anomic/kelondro/index/ObjectIndex.java, source/de/anomic/kelondro/index/ObjectIndexCache.java, source/de/anomic/kelondro/index/ObjectIndexDaemon.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/Relations.java, source/de/anomic/kelondro/table/SQLTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/yacy/yacyNewsDB.java
Tue Mar 10 21:52:10 CET 2009
by orbiter
added also an export and delete-feature to the URLAnalysis. This completes the clean-up feature for URLs. To do a complete clean-up of the url database, start the following:
java -Xmx1000m -cp classes de.anomic.data.URLAnalysis -incollection DATA/INDEX/freeworld/TEXT/RICOLLECTION used.dump
java -Xmx1000m -cp classes de.anomic.data.URLAnalysis -diffurlcol DATA/INDEX/freeworld/TEXT used.dump diffurlcol.dump
java -Xmx1000m -cp classes de.anomic.data.URLAnalysis -export DATA/INDEX/freeworld/TEXT xml urls.xml diffurlcol.dump
java -Xmx1000m -cp classes de.anomic.data.URLAnalysis -delete DATA/INDEX/freeworld/TEXT diffurlcol.dump

The export-feature is optional, the purpose of that function is to provide a back-up function for URLs to be deleted. The export function can also be used to create html files with embedded links and simple text-files. Simply replace the 'xml' word with 'html' or 'text'. The last argument in the cann, the diffurlcol.dump value, can also be omitted. This will cause that the complete URL database is exported. This is an alternative to the Web-Interface based export function.

The delete-feature is the only destructive method of the four presented here. Please use it with care. It is better to make a back-up of the url database files before starting the deletion.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5694 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/URLAnalysis.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/plasma/plasmaWordIndex.java
Tue Mar 10 14:38:40 CET 2009
by orbiter
added next tool for url analysis: check for references, that occur in the URL-DB but not in the RICOLLECTIONS
to use this, you must user the -incollection command before (see SVN 5687) and you need a 
used.dump file that has been produced with that process.

Now you can use that file, to do a URL-hash compare with the urls in the URL-DB. To do that, execute
java -Xmx1000m -cp classes de.anomic.data.URLAnalysis -diffurlcol DATA/INDEX/freeworld/TEXT used.dump diffurlcol.dump
or use different names for the dump files or more memory.

As a result, you get the file diffurlcol.dump which contains all the url hashes that occur in the URL database, but not in the collections.
The file has the format
{hash-12}*
that means: 12 byte long hashes are listed without any separation.

The next step could be to process this file and delete all these URLs with the computed hashes, or to export them before deletion.



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5692 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/URLAnalysis.java, source/de/anomic/kelondro/index/HandleSet.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/plasma/plasmaWordIndex.java, source/yacy.java
Sun Mar 08 22:37:17 CET 2009
by orbiter
- refactoring of IntegerHandleIndex and LongHandleIndex (better method names)
- fix for problem in httpdFileHandler: mising close of open Files if tempate cache was disabled
- more memory for DHT selection required
- stub for URL reference hash statistics in index collections

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5682 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/URLAnalysis.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/kelondro/blob/BLOBHeap.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/plasma/plasmaSwitchboard.java
Fri Mar 06 16:53:20 CET 2009
by orbiter
simplification of (internal) query process / refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5671 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Blog.java, htroot/Bookmarks.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImport_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/xbel/xbel.java, htroot/cytag.java, htroot/favicon.ico, htroot/rct_p.java, htroot/yacy/query.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRequestHeader.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/text/MetadataRowContainer.java, source/de/anomic/kelondro/util/DateFormatter.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/server/serverObjects.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeed.java, source/yacy.java
Mon Mar 02 12:04:13 CET 2009
by orbiter
more refactoring of kelondro.text / deleted de.anomic.index

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5664 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/BlacklistCleaner_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/api/blacklists_p.java, htroot/api/yacydoc.java, htroot/sharedBlacklist_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/listManager.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/text/AbstractBlacklist.java, source/de/anomic/kelondro/text/Blacklist.java, source/de/anomic/kelondro/text/DefaultBlacklist.java, source/de/anomic/kelondro/text/Document.java, source/de/anomic/kelondro/text/IndexCache.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexCollection.java, source/de/anomic/kelondro/text/MetadataRepository.java, source/de/anomic/kelondro/text/MetadataRowContainer.java, source/de/anomic/kelondro/text/ReferenceContainerArray.java, source/de/anomic/kelondro/text/ReferenceContainerCache.java, source/de/anomic/kelondro/text/ReferenceContainerOrder.java, source/de/anomic/kelondro/text/URLMetadata.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/ymage/ymageOSM.java, source/yacy.java
Mon Mar 02 11:00:32 CET 2009
by orbiter
more refactoring of indexer and kelondro classes;
- integrating the indexer into kelondro as package 'text'
- renaming of classes in kelondro.index

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5663 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/dbtest.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/index/URLMetadata.java, source/de/anomic/index/URLMetadataRepository.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainerBLOBArray.java, source/de/anomic/index/indexContainerCache.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/IntegerHandleIndex.java, source/de/anomic/kelondro/index/LongHandleIndex.java, source/de/anomic/kelondro/index/ObjectArray.java, source/de/anomic/kelondro/index/ObjectArrayCache.java, source/de/anomic/kelondro/index/ObjectIndexCache.java, source/de/anomic/kelondro/table/CachedRecords.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FixedWidthArray.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/FlexWidthArray.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/text/Index.java, source/de/anomic/kelondro/text/IndexCache.java, source/de/anomic/kelondro/text/IndexCell.java, source/de/anomic/kelondro/text/IndexReader.java, source/de/anomic/kelondro/text/Phrase.java, source/de/anomic/kelondro/text/Reference.java, source/de/anomic/kelondro/text/ReferenceContainer.java, source/de/anomic/kelondro/text/ReferenceOrder.java, source/de/anomic/kelondro/text/ReferenceRow.java, source/de/anomic/kelondro/text/ReferenceVars.java, source/de/anomic/kelondro/text/Word.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeed.java, source/yacy.java
Mon Mar 02 00:58:14 CET 2009
by orbiter
refactoring of index package: better names for the classes (to be continued)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5661 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/ViewFile.java, htroot/api/yacydoc.java, htroot/yacy/crawlReceipt.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/index/Reference.java, source/de/anomic/index/ReferenceContainer.java, source/de/anomic/index/ReferenceOrder.java, source/de/anomic/index/ReferenceRow.java, source/de/anomic/index/ReferenceVars.java, source/de/anomic/index/ReverseIndex.java, source/de/anomic/index/ReverseIndexCache.java, source/de/anomic/index/ReverseIndexCell.java, source/de/anomic/index/ReverseIndexReader.java, source/de/anomic/index/URLMetadata.java, source/de/anomic/index/URLMetadataRepository.java, source/de/anomic/index/Word.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainerBLOBArray.java, source/de/anomic/index/indexContainerCache.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeed.java, source/yacy.java
Tue Feb 24 11:40:20 CET 2009
by orbiter
added new row interator in kelondro tables files that enumerates rows
without an order by the primary key. The result is a very fast enumeration of the Eco table data structure. Other table data types are not affected.
The new enumerator is used for the url export function that can be accessed from the online interface (Index Administration -> URL References -> Export). This export should now be much faster, if all url database files are from type Eco
The new enumeration is also used at other functions in YaCy, i.e. the initialization of the crawl balancer and the initialization of YaCy News.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5647 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/blob/Cache.java, source/de/anomic/kelondro/index/IntBytesMap.java, source/de/anomic/kelondro/index/ObjectIndex.java, source/de/anomic/kelondro/index/RAMIndex.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/order/StackIterator.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/SQLTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/yacy/yacyNewsDB.java
Thu Feb 19 17:24:46 CET 2009
by orbiter
- refactoring of the http client
- added a protection against memory leaks for the access tracker

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5621 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/Connections_p.java, htroot/Network.java, htroot/ViewFile.java, htroot/api/util/getpageinfo_p.java, htroot/sharedBlacklist_p.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/httpClient.java, source/de/anomic/http/httpResponse.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/net/natLib.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverAccessTracker.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Tue Feb 17 10:12:47 CET 2009
by orbiter
more bugfixes as recommendet by findbugs

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5619 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Threaddump_p.java, htroot/api/util/ynetSearch.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/htmlFilter/htmlFilterCharacterCoding.java, source/de/anomic/index/indexCell.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/blob/BLOBCompressor.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/Cache.java, source/de/anomic/kelondro/blob/Gap.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/table/ChunkIterator.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyTray.java
Mon Feb 16 00:35:59 CET 2009
by orbiter
- better memory protection in kelondro caches: computation of needed memory for cache grow
- removed excessive gc calls
- step to 16 vertical DHT partitions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5611 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.network.freeworld.unit, source/de/anomic/kelondro/blob/Cache.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/table/CachedRecords.java, source/de/anomic/kelondro/util/MemoryControl.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/yacy/dht/Dispatcher.java
Sat Feb 14 14:12:08 CET 2009
by lotus
added UPnP support
YaCy can now automatically forward ports on home routers
off by default

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5609 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.xml, defaults/yacy.init, htroot/ConfigBasic.java, lib/commons-jxpath-1.1.License, lib/commons-jxpath-1.1.jar, lib/sbbi-upnplib-1.0.4.License, lib/sbbi-upnplib-1.0.4.jar, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/tools/UPnP.java, yacy.logging
Wed Feb 11 14:31:10 CET 2009
by orbiter
more memory leak fixing hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5599 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/ImporterManager.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/diff.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterInputStream.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/index/BinSearch.java, source/de/anomic/kelondro/index/BytesIntMap.java, source/de/anomic/kelondro/index/BytesLongMap.java, source/de/anomic/kelondro/table/SQLTable.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverCore.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/logging/ConsoleOutErrHandler.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/ymage/AnimGifEncoder.java
Wed Feb 11 13:58:04 CET 2009
by apfelmaennchen
YaCy-UI:
- some themes to get started

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5598 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/themes/base/images/theme.png, htroot/yacy/ui/css/themes/cupertino/images/theme.png, htroot/yacy/ui/css/themes/darkness/images/theme.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_flat_30_cccccc_40x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_flat_50_5c5c5c_40x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_20_555555_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_40_0078a3_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_glass_40_ffc73d_1x400.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_gloss-wave_25_333333_500x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_highlight-soft_80_eeeeee_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_inset-soft_25_000000_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-bg_inset-soft_30_f58400_1x100.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_4b8e0b_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_a83300_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_cccccc_256x240.png, htroot/yacy/ui/css/themes/darkness/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/darkness/ui.accordion.css, htroot/yacy/ui/css/themes/darkness/ui.all.css, htroot/yacy/ui/css/themes/darkness/ui.base.css, htroot/yacy/ui/css/themes/darkness/ui.core.css, htroot/yacy/ui/css/themes/darkness/ui.datepicker.css, htroot/yacy/ui/css/themes/darkness/ui.dialog.css, htroot/yacy/ui/css/themes/darkness/ui.progressbar.css, htroot/yacy/ui/css/themes/darkness/ui.resizable.css, htroot/yacy/ui/css/themes/darkness/ui.slider.css, htroot/yacy/ui/css/themes/darkness/ui.tabs.css, htroot/yacy/ui/css/themes/darkness/ui.theme.css, htroot/yacy/ui/css/themes/humanity/images/theme.png, htroot/yacy/ui/css/themes/redmond/images/theme.png, htroot/yacy/ui/css/themes/smoothness/images/theme.png
Wed Feb 11 13:50:42 CET 2009
by apfelmaennchen
YaCy-UI:
- updated to jquery 1.3.1
- updated to jquery-ui 1.6rc6
- added support for themes (not yet persistent but otherwise working well)
- deleted some obsolete files

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5597 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/base.css, htroot/yacy/ui/css/themes/base/images/ui-bg_flat_55_999999_40x100.png, htroot/yacy/ui/css/themes/base/images/ui-bg_flat_75_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/base/images/ui-bg_glass_45_0078ae_1x400.png, htroot/yacy/ui/css/themes/base/images/ui-bg_glass_55_f8da4e_1x400.png, htroot/yacy/ui/css/themes/base/images/ui-bg_glass_75_79c9ec_1x400.png, htroot/yacy/ui/css/themes/base/images/ui-bg_gloss-wave_45_e14f1c_500x100.png, htroot/yacy/ui/css/themes/base/images/ui-bg_gloss-wave_50_6eac2c_500x100.png, htroot/yacy/ui/css/themes/base/images/ui-bg_gloss-wave_75_2191c0_500x100.png, htroot/yacy/ui/css/themes/base/images/ui-bg_inset-hard_100_fcfdfd_1x100.png, htroot/yacy/ui/css/themes/base/images/ui-icons_0078ae_256x240.png, htroot/yacy/ui/css/themes/base/images/ui-icons_056b93_256x240.png, htroot/yacy/ui/css/themes/base/images/ui-icons_d8e7f3_256x240.png, htroot/yacy/ui/css/themes/base/images/ui-icons_e0fdff_256x240.png, htroot/yacy/ui/css/themes/base/images/ui-icons_f5e175_256x240.png, htroot/yacy/ui/css/themes/base/images/ui-icons_f7a50d_256x240.png, htroot/yacy/ui/css/themes/base/images/ui-icons_fcd113_256x240.png, htroot/yacy/ui/css/themes/base/ui.accordion.css, htroot/yacy/ui/css/themes/base/ui.all.css, htroot/yacy/ui/css/themes/base/ui.base.css, htroot/yacy/ui/css/themes/base/ui.core.css, htroot/yacy/ui/css/themes/base/ui.datepicker.css, htroot/yacy/ui/css/themes/base/ui.dialog.css, htroot/yacy/ui/css/themes/base/ui.progressbar.css, htroot/yacy/ui/css/themes/base/ui.resizable.css, htroot/yacy/ui/css/themes/base/ui.slider.css, htroot/yacy/ui/css/themes/base/ui.tabs.css, htroot/yacy/ui/css/themes/base/ui.theme.css, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_diagonals-small_0_aaaaaa_40x40.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_diagonals-thick_15_444444_40x40.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_100_f0f0f0_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_50_99c2ff_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_55_fbf5d0_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_80_e6e6e6_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_highlight-hard_100_f9f9f9_1x100.png, htroot/yacy/ui/css/themes/cupertino/images/ui-bg_highlight-soft_100_e7eef3_1x100.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_2694e8_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_72a7cf_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_888888_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/cupertino/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/cupertino/ui.accordion.css, htroot/yacy/ui/css/themes/cupertino/ui.all.css, htroot/yacy/ui/css/themes/cupertino/ui.base.css, htroot/yacy/ui/css/themes/cupertino/ui.core.css, htroot/yacy/ui/css/themes/cupertino/ui.datepicker.css, htroot/yacy/ui/css/themes/cupertino/ui.dialog.css, htroot/yacy/ui/css/themes/cupertino/ui.progressbar.css, htroot/yacy/ui/css/themes/cupertino/ui.resizable.css, htroot/yacy/ui/css/themes/cupertino/ui.slider.css, htroot/yacy/ui/css/themes/cupertino/ui.tabs.css, htroot/yacy/ui/css/themes/cupertino/ui.theme.css, htroot/yacy/ui/css/themes/humanity/images/ui-bg_flat_75_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_100_f5f0e5_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_25_cb842e_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_glass_70_ede4d4_1x400.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_100_f4f0ec_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_65_fee4bd_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_highlight-hard_75_f5f5b5_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-bg_inset-soft_100_f4f0ec_1x100.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_c47a23_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_cb672b_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_f08000_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_f35f07_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_ff7519_256x240.png, htroot/yacy/ui/css/themes/humanity/images/ui-icons_ffffff_256x240.png, htroot/yacy/ui/css/themes/humanity/ui.accordion.css, htroot/yacy/ui/css/themes/humanity/ui.all.css, htroot/yacy/ui/css/themes/humanity/ui.base.css, htroot/yacy/ui/css/themes/humanity/ui.core.css, htroot/yacy/ui/css/themes/humanity/ui.datepicker.css, htroot/yacy/ui/css/themes/humanity/ui.dialog.css, htroot/yacy/ui/css/themes/humanity/ui.progressbar.css, htroot/yacy/ui/css/themes/humanity/ui.resizable.css, htroot/yacy/ui/css/themes/humanity/ui.slider.css, htroot/yacy/ui/css/themes/humanity/ui.tabs.css, htroot/yacy/ui/css/themes/humanity/ui.theme.css, htroot/yacy/ui/css/themes/redmond/images/ui-bg_flat_0_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_flat_55_fbec88_40x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_75_d0e5f5_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_85_dfeffc_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_gloss-wave_55_5c9ccc_500x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_inset-hard_100_f5f8f9_1x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-bg_inset-hard_100_fcfdfd_1x100.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_217bc0_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_469bdd_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_6da8d5_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_d8e7f3_256x240.png, htroot/yacy/ui/css/themes/redmond/images/ui-icons_f9bd01_256x240.png, htroot/yacy/ui/css/themes/redmond/ui.accordion.css, htroot/yacy/ui/css/themes/redmond/ui.all.css, htroot/yacy/ui/css/themes/redmond/ui.base.css, htroot/yacy/ui/css/themes/redmond/ui.core.css, htroot/yacy/ui/css/themes/redmond/ui.datepicker.css, htroot/yacy/ui/css/themes/redmond/ui.dialog.css, htroot/yacy/ui/css/themes/redmond/ui.progressbar.css, htroot/yacy/ui/css/themes/redmond/ui.resizable.css, htroot/yacy/ui/css/themes/redmond/ui.slider.css, htroot/yacy/ui/css/themes/redmond/ui.tabs.css, htroot/yacy/ui/css/themes/redmond/ui.theme.css, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_flat_0_aaaaaa_40x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_flat_75_ffffff_40x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_55_fbf9ee_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_65_ffffff_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_75_dadada_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_75_e6e6e6_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_glass_95_fef1ec_1x400.png, htroot/yacy/ui/css/themes/smoothness/images/ui-bg_highlight-soft_75_cccccc_1x100.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_222222_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_2e83ff_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_454545_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_888888_256x240.png, htroot/yacy/ui/css/themes/smoothness/images/ui-icons_cd0a0a_256x240.png, htroot/yacy/ui/css/themes/smoothness/ui.accordion.css, htroot/yacy/ui/css/themes/smoothness/ui.all.css, htroot/yacy/ui/css/themes/smoothness/ui.base.css, htroot/yacy/ui/css/themes/smoothness/ui.core.css, htroot/yacy/ui/css/themes/smoothness/ui.datepicker.css, htroot/yacy/ui/css/themes/smoothness/ui.dialog.css, htroot/yacy/ui/css/themes/smoothness/ui.progressbar.css, htroot/yacy/ui/css/themes/smoothness/ui.resizable.css, htroot/yacy/ui/css/themes/smoothness/ui.slider.css, htroot/yacy/ui/css/themes/smoothness/ui.tabs.css, htroot/yacy/ui/css/themes/smoothness/ui.theme.css, htroot/yacy/ui/index.html, htroot/yacy/ui/js/jquery-1.3.1.min.js, htroot/yacy/ui/js/jquery.ui.all.min.js, htroot/yacy/ui/sidebar/sidebar_1.html, htroot/yacy/ui/sidebar/sidebar_2.html, htroot/yacy/ui/yacyui-admin.html, htroot/yacy/ui/yacyui-bookmarks.html, htroot/yacy/ui/yacyui-search.html
Tue Feb 10 01:06:59 CET 2009
by orbiter
replaced old DHT transmission method with new method. Many things have changed! some of them:
- after a index selection is made, the index is splitted into its vertical components
- from differrent index selctions the splitted components can be accumulated before they are placed into the transmission queue
- each splitted chunk gets its own transmission thread
- multiple transmission threads are started concurrently
- the process can be monitored with the blocking queue servlet
To implement that, a new package de.anomic.yacy.dht was created. Some old files have been removed.
The new index distribution model using a vertical DHT was implemented. An abstraction of this model
is implemented in the new dht package as interface. The freeworld network has now a configuration
of two vertial partitions; sixteen partitions are planned and will be configured if the process is bug-free.
This modification has three main targets:
- enhance the DHT transmission speed
- with a vertical DHT, a search will speed up. With two partitions, two times. With sixteen, sixteen times.
- the vertical DHT will apply a semi-dht for URLs, and peers will receive a fraction of the overall URLs they received before.
  with two partitions, the fractions will be halve. With sixteen partitions, a 1/16 of the previous number of URLs.
BE CAREFULL, THIS IS A MAJOR CODE CHANGE, POSSIBLY FULL OF BUGS AND HARMFUL THINGS.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5586 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/Blog.java, htroot/Bookmarks.java, htroot/ConfigProfile_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexTransfer_p.java, htroot/MessageSend_p.java, htroot/Network.java, htroot/News.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/Wiki.java, htroot/api/bookmarks/posts/add_p.java, htroot/yacy/hello.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/index/indexCell.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainerBLOBArray.java, source/de/anomic/index/indexContainerCache.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRI.java, source/de/anomic/index/indexRIReader.java, source/de/anomic/kelondro/io/EcoFS.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/dht/Dispatcher.java, source/de/anomic/yacy/dht/FlatWordPartitionScheme.java, source/de/anomic/yacy/dht/PartitionScheme.java, source/de/anomic/yacy/dht/PeerSelection.java, source/de/anomic/yacy/dht/Transmission.java, source/de/anomic/yacy/dht/VerticalWordPartitionScheme.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Fri Feb 06 16:06:19 CET 2009
by orbiter
added new cybertag-tracking feature that was inspired by itgrl
from the forum discussion in
http://forum.yacy-websuche.de/viewtopic.php?p=12612#p12612

The feature will provide two basic entities:
- you can integrate image links which point to your yacy installation anywhere in the web.
  the image can be loaded with
  <img src="http://<yourpeer>:<yourport>/cytag.png?icon=invisible&nick=<yournickname_or_community_id>&tag=<anything>">
  This will place a invisible 1-pixel image. If you change the icon=invisible to icon=redpill, you will see a red pill
  Use this, to track your activity in the web.
- you can view your tracks at
  http://localhost:8080/Tracks.html
- There is a public api to your tracks at
  http://localhost:8080/api/tracks_p.json
  which needs authentication


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5581 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.xml, htroot/ConfigBasic.html, htroot/Trails.html, htroot/Trails.java, htroot/api/trail_p.java, htroot/api/trail_p.json, htroot/cytag.java, htroot/env/grafics/invisible.png, htroot/env/grafics/redpillmini.png, htroot/index.html, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/kelondro/order/DateFormatter.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/servletProperties.java
Fri Feb 06 15:45:56 CET 2009
by auron_x
- more XHTML-validation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5580 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.html, htroot/CrawlProfileEditor_p.html, htroot/CrawlResults.html, htroot/CrawlStart_p.html, htroot/Network.html, htroot/Surftips.html, htroot/WatchCrawler_p.html, htroot/Wiki.html, htroot/compare_yacy.html, htroot/terminal_p.html, htroot/yacy/ui/index.html, htroot/yacyinteractive.html, htroot/yacysearch.html, htroot/yacysearchitem.html
Thu Feb 05 16:15:13 CET 2009
by orbiter
- fixes to doc, ppt, xls parser: better title
- fixes to httpd server response header generation
- fixes to a server date computation bug
- new Button in indexControl to view content of url in ViewFile


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5576 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlURLs_p.html, htroot/ViewFile.java, htroot/api/yacydoc.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpResponseHeader.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/kelondro/order/DateFormatter.java, source/de/anomic/plasma/parser/doc/docParser.java, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/xls/xlsParser.java
Tue Feb 03 14:04:02 CET 2009
by orbiter
- renamed interactivesearch to yacyinteractive
- added a configuration option to set the pop up page in Config Appearance
- added a minimized header option to yacyinteractive
- fixed a bug in yacysearch: default values when no query is done


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5569 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.html, htroot/ConfigAppearance_p.java, htroot/env/templates/header.template, htroot/index.java, htroot/yacyinteractive.html, htroot/yacyinteractive.java, htroot/yacysearch.html, htroot/yacysearch.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/yacy.java
Sat Jan 31 02:06:56 CET 2009
by orbiter
moved logging partially to kelondro

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5545 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlogComments.java, htroot/CacheResource_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexTransfer_p.java, htroot/Messages_p.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/WatchCrawler_p.java, htroot/api/util/getpageinfo_p.java, htroot/sharedBlacklist_p.java, htroot/yacy/message.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/translator.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/JakartaCommonsHttpResponse.java, source/de/anomic/http/httpChunkedOutputStream.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/io/BufferedEcoFS.java, source/de/anomic/kelondro/io/EcoFS.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexWidthArray.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/kelondro/util/AttrSeq.java, source/de/anomic/kelondro/util/FileUtils.java, source/de/anomic/kelondro/util/MemoryControl.java, source/de/anomic/kelondro/util/NamePrefixThreadFactory.java, source/de/anomic/net/natLib.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/sevenzip/sevenzipParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverCachedFileOutputStream.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverSystem.java, source/de/anomic/tools/Formatter.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/tools/nxTools.java, source/de/anomic/yacy/logging/ConsoleOutErrHandler.java, source/de/anomic/yacy/logging/ConsoleOutHandler.java, source/de/anomic/yacy/logging/LogalizerHandler.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFile.java, source/de/anomic/yacy/yacyAccessible.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/migration.java, source/yacy.java, yacy.logging
Sat Jan 31 00:33:47 CET 2009
by orbiter
refactoring of logging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5544 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/LogStatistics_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/message.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/de/anomic/crawler/AbstractImporter.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/ImporterManager.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/translator.java, source/de/anomic/http/AcceptEverythingTrustManager.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/HttpConnectionInfo.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/ProxyLogFormatter.java, source/de/anomic/http/httpResponseHeader.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainerRAMHeap.java, source/de/anomic/index/indexDocumentCache.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/table/AbstractRecords.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/FlexWidthArray.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/kelondro/util/Log.java, source/de/anomic/kelondro/util/MemoryControl.java, source/de/anomic/language/identification/LanguageStatistics.java, source/de/anomic/language/identification/LanguageStatisticsHolder.java, source/de/anomic/plasma/LogParser.java, source/de/anomic/plasma/parser/AbstractParser.java, source/de/anomic/plasma/parser/Parser.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/sevenzip/SZParserExtractCallback.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverAbstractThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/server/serverSwitchAbstractAction.java, source/de/anomic/server/serverSwitchAction.java, source/de/anomic/server/serverSystem.java, source/de/anomic/server/serverThread.java, source/de/anomic/tools/consoleInterface.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/tools/gzip.java, source/de/anomic/tools/tarTools.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/logging/GuiHandler.java, source/de/anomic/yacy/logging/LogalizerHandler.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/ymageOSM.java, source/migration.java, source/yacy.java
Fri Jan 30 23:44:20 CET 2009
by orbiter
(almost) completed partition of classes in kelondro

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5543 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexImport_p.java, htroot/IndexTransfer_p.java, htroot/PerformanceMemory_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/http/httpChunkedInputStream.java, source/de/anomic/http/httpChunkedOutputStream.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/index/indexAbstractReferenceBlacklist.java, source/de/anomic/index/indexCell.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/BLOBCompressor.java, source/de/anomic/kelondro/blob/BLOBHeap.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/Cache.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/MapDataMining.java, source/de/anomic/kelondro/blob/MapView.java, source/de/anomic/kelondro/blob/ObjectBuffer.java, source/de/anomic/kelondro/index/Column.java, source/de/anomic/kelondro/index/RAMIndex.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/io/AbstractRandomAccess.java, source/de/anomic/kelondro/io/BufferedRandomAccess.java, source/de/anomic/kelondro/io/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/order/AbstractOrder.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/order/ByteOrder.java, source/de/anomic/kelondro/order/MergeIterator.java, source/de/anomic/kelondro/order/NaturalOrder.java, source/de/anomic/kelondro/order/Order.java, source/de/anomic/kelondro/order/RotateIterator.java, source/de/anomic/kelondro/table/AbstractRecords.java, source/de/anomic/kelondro/table/CachedRecords.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/FlexWidthArray.java, source/de/anomic/kelondro/table/FullRecords.java, source/de/anomic/kelondro/table/Hashtable.java, source/de/anomic/kelondro/table/Relations.java, source/de/anomic/kelondro/table/SQLTable.java, source/de/anomic/kelondro/table/SplitTable.java, source/de/anomic/kelondro/table/Stack.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/kelondro/util/AttrSeq.java, source/de/anomic/kelondro/util/ByteArray.java, source/de/anomic/kelondro/util/ByteBuffer.java, source/de/anomic/kelondro/util/CompressedHashMap.java, source/de/anomic/kelondro/util/MemoryControl.java, source/de/anomic/kelondro/util/ObjectSpace.java, source/de/anomic/kelondro/util/ScoreCluster.java, source/de/anomic/kelondro/util/SetTools.java, source/de/anomic/kelondro/util/SortStack.java, source/de/anomic/kelondro/util/SortStore.java, source/de/anomic/kelondro/util/XMLTables.java, source/de/anomic/kelondro/util/kelondroException.java, source/de/anomic/kelondro/util/kelondroOutOfLimitsException.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/ymage/ymageMatrix.java, source/yacy.java
Fri Jan 30 23:08:08 CET 2009
by orbiter
more refactoring of kelondro:
- seperated BLOB from table classes
- renamed 'coding' package to 'order'

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5542 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Blog.java, htroot/Bookmarks.java, htroot/ConfigAccounts_p.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImport_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/PerformanceMemory_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/User.java, htroot/ViewProfile.java, htroot/WebStructurePicture_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/xbel/xbel.java, htroot/rct_p.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRequestHeader.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexCell.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerBLOBArray.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/index/indexContainerRAMHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRI.java, source/de/anomic/index/indexRIReader.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/index/indexWord.java, source/de/anomic/kelondro/blob/BLOB.java, source/de/anomic/kelondro/blob/BLOBArray.java, source/de/anomic/kelondro/blob/BLOBCompressor.java, source/de/anomic/kelondro/blob/BLOBHeap.java, source/de/anomic/kelondro/blob/BLOBHeapModifier.java, source/de/anomic/kelondro/blob/BLOBTree.java, source/de/anomic/kelondro/blob/Gap.java, source/de/anomic/kelondro/blob/HeapReader.java, source/de/anomic/kelondro/blob/HeapWriter.java, source/de/anomic/kelondro/index/BinSearch.java, source/de/anomic/kelondro/index/BytesIntMap.java, source/de/anomic/kelondro/index/BytesLongMap.java, source/de/anomic/kelondro/index/IntBytesMap.java, source/de/anomic/kelondro/index/ObjectIndex.java, source/de/anomic/kelondro/index/RAMIndex.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/io/RandomAccessRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/kelondro/kelondroObjectBuffer.java, source/de/anomic/kelondro/kelondroRelations.java, source/de/anomic/kelondro/kelondroRotateIterator.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroStack.java, source/de/anomic/kelondro/order/AbstractOrder.java, source/de/anomic/kelondro/order/Base64Order.java, source/de/anomic/kelondro/order/Bitfield.java, source/de/anomic/kelondro/order/ByteOrder.java, source/de/anomic/kelondro/order/CloneableIterator.java, source/de/anomic/kelondro/order/CloneableMapIterator.java, source/de/anomic/kelondro/order/Coding.java, source/de/anomic/kelondro/order/DateFormatter.java, source/de/anomic/kelondro/order/Digest.java, source/de/anomic/kelondro/order/MicroDate.java, source/de/anomic/kelondro/order/NaturalOrder.java, source/de/anomic/kelondro/order/kelondroMergeIterator.java, source/de/anomic/kelondro/order/kelondroOrder.java, source/de/anomic/kelondro/table/AbstractRecords.java, source/de/anomic/kelondro/table/CachedRecords.java, source/de/anomic/kelondro/table/ChunkIterator.java, source/de/anomic/kelondro/table/EcoTable.java, source/de/anomic/kelondro/table/FixedWidthArray.java, source/de/anomic/kelondro/table/FlexTable.java, source/de/anomic/kelondro/table/FlexWidthArray.java, source/de/anomic/kelondro/table/FullRecords.java, source/de/anomic/kelondro/table/Hashtable.java, source/de/anomic/kelondro/table/Node.java, source/de/anomic/kelondro/table/RecordHandle.java, source/de/anomic/kelondro/table/Tree.java, source/de/anomic/kelondro/tools/ByteArray.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverObjects.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyURL.java, source/genpw.java, source/migration.java, source/yacy.java
Fri Jan 30 16:33:00 CET 2009
by orbiter
refactoring of kelondro database classes
(to be continued)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5540 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Blog.java, htroot/Bookmarks.java, htroot/ConfigAccounts_p.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexImport_p.java, htroot/IndexTransfer_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/PerformanceMemory_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/User.java, htroot/ViewProfile.java, htroot/WebStructurePicture_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/status_p.java, htroot/rct_p.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpChunkedInputStream.java, source/de/anomic/http/httpChunkedOutputStream.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRequestHeader.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexAbstractReferenceBlacklist.java, source/de/anomic/index/indexCell.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerBLOBArray.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/index/indexContainerRAMHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/index/indexWord.java, source/de/anomic/kelondro/coding/AbstractOrder.java, source/de/anomic/kelondro/coding/Base64Order.java, source/de/anomic/kelondro/coding/Bitfield.java, source/de/anomic/kelondro/coding/ByteOrder.java, source/de/anomic/kelondro/coding/Coding.java, source/de/anomic/kelondro/coding/DateFormatter.java, source/de/anomic/kelondro/coding/Digest.java, source/de/anomic/kelondro/coding/MicroDate.java, source/de/anomic/kelondro/coding/NaturalOrder.java, source/de/anomic/kelondro/coding/kelondroOrder.java, source/de/anomic/kelondro/index/Array.java, source/de/anomic/kelondro/index/BinSearch.java, source/de/anomic/kelondro/index/BytesIntMap.java, source/de/anomic/kelondro/index/BytesLongMap.java, source/de/anomic/kelondro/index/Column.java, source/de/anomic/kelondro/index/ObjectIndex.java, source/de/anomic/kelondro/index/RAMIndex.java, source/de/anomic/kelondro/index/Row.java, source/de/anomic/kelondro/index/RowCollection.java, source/de/anomic/kelondro/index/RowSet.java, source/de/anomic/kelondro/io/AbstractIOChunks.java, source/de/anomic/kelondro/io/AbstractRandomAccess.java, source/de/anomic/kelondro/io/BufferedIOChunks.java, source/de/anomic/kelondro/io/BufferedRandomAccess.java, source/de/anomic/kelondro/io/CachedRandomAccess.java, source/de/anomic/kelondro/io/ChannelRandomAccess.java, source/de/anomic/kelondro/io/FileRandomAccess.java, source/de/anomic/kelondro/io/IOChunksInterface.java, source/de/anomic/kelondro/io/RandomAccessIOChunks.java, source/de/anomic/kelondro/io/RandomAccessInterface.java, source/de/anomic/kelondro/io/RandomAccessRecords.java, source/de/anomic/kelondro/io/kelondroBufferedEcoFS.java, source/de/anomic/kelondro/io/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBLOB.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroBLOBCompressor.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBHeapModifier.java, source/de/anomic/kelondro/kelondroBLOBHeapReader.java, source/de/anomic/kelondro/kelondroBLOBHeapWriter.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFixedWidthArray.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFlexWidthArray.java, source/de/anomic/kelondro/kelondroFullRecords.java, source/de/anomic/kelondro/kelondroHashtable.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroObjectBuffer.java, source/de/anomic/kelondro/kelondroRelations.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroStack.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/kelondro/tools/ByteArray.java, source/de/anomic/kelondro/tools/ByteBuffer.java, source/de/anomic/kelondro/tools/CompressedHashMap.java, source/de/anomic/kelondro/tools/MemoryControl.java, source/de/anomic/kelondro/tools/ScoreCluster.java, source/de/anomic/kelondro/tools/SetTools.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/ymage/ymageMatrix.java, source/genpw.java, source/migration.java, source/yacy.java
Fri Jan 30 15:48:11 CET 2009
by orbiter
refactoring of kelondro classes:
kelondro shall become independent from other packages.
moved bytebuffer, date and memory to kelondro

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5539 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Blog.java, htroot/Bookmarks.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexImport_p.java, htroot/IndexTransfer_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/PerformanceMemory_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/status_p.java, htroot/rct_p.java, htroot/yacy/query.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/dbtest.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/http/httpChunkedInputStream.java, source/de/anomic/http/httpChunkedOutputStream.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRequestHeader.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroBLOBHeapModifier.java, source/de/anomic/kelondro/kelondroBLOBHeapReader.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBufferedRA.java, source/de/anomic/kelondro/kelondroByteBuffer.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDate.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroMemory.java, source/de/anomic/kelondro/kelondroNaturalOrder.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/logging/serverLog.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/tools/mediawikiIndex.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/ymage/ymageMatrix.java, source/yacy.java
Wed Jan 28 01:15:43 CET 2009
by apfelmaennchen
- added two dialogs to manage bookmark tags in YaCy-UI
- fixed renameTag() in bookmarksDB
- added /api/bookmarks/tags/addTag.xml
- added /api/bookmarks/tags/editTag.xml

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5525 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/api/bookmarks/tags/addTag.java, htroot/api/bookmarks/tags/addTag.xml, htroot/api/bookmarks/tags/editTag.java, htroot/api/bookmarks/tags/editTag.xml, htroot/yacy/ui/css/base.css, htroot/yacy/ui/index.html, htroot/yacy/ui/yacyui-bookmarks.html, htroot/yacy/ui/yacyui-welcome.html, source/de/anomic/data/bookmarksDB.java
Tue Jan 27 19:17:59 CET 2009
by apfelmaennchen
- some additional icons for YaCy-UI
- added license (readme.txt) to exiting icon sets

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5524 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/img-1/readme.txt, htroot/yacy/ui/img-2/readme.txt, htroot/yacy/ui/img/tags/readme.txt, htroot/yacy/ui/img/tags/tag_blue.png, htroot/yacy/ui/img/tags/tag_blue_add.png, htroot/yacy/ui/img/tags/tag_blue_delete.png, htroot/yacy/ui/img/tags/tag_blue_edit.png, htroot/yacy/ui/img/tags/tag_green.png, htroot/yacy/ui/img/tags/tag_orange.png, htroot/yacy/ui/img/tags/tag_pink.png, htroot/yacy/ui/img/tags/tag_purple.png, htroot/yacy/ui/img/tags/tag_red.png, htroot/yacy/ui/img/tags/tag_yellow.png
Mon Jan 26 19:38:58 CET 2009
by apfelmaennchen
Major update to YaCy-UI...still not perfect...but I thought I share my progress :-)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5519 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.json, htroot/yacy/ui/css/base.css, htroot/yacy/ui/index.html, htroot/yacy/ui/js/jquery.treeview.async.js, htroot/yacy/ui/sidebar/sidebar_1.html, htroot/yacy/ui/yacyui-admin.html, htroot/yacy/ui/yacyui-bookmarks.html, htroot/yacy/ui/yacyui-search.html, htroot/yacy/ui/yacyui-welcome.html
Fri Jan 23 16:32:27 CET 2009
by orbiter
added more methods to control the vertical DHT (not yet active .. )

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5514 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.network.freeworld.unit, defaults/yacy.network.intranet.unit, defaults/yacy.network.readme, defaults/yacy.network.webportal.unit, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Thu Jan 22 01:03:54 CET 2009
by orbiter
removed distinction between DHT-in and DHT-out. This is necessary to make room for the new cell data structure, which cannot use this this distinction in the first place, but will enable the same meaning with different mechanisms (segments, later)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5511 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, htroot/PerformanceQueues_p.xml, htroot/WatchCrawler_p.java, htroot/api/queues_p.java, htroot/api/status_p.java, htroot/api/status_p.xml, htroot/js/WatchCrawler.js, htroot/yacy/transferRWI.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/kelondro/kelondroBufferedEcoFS.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java
Wed Jan 21 19:23:37 CET 2009
by orbiter
preparations to integrate the new 'cell' index data structure
(this commit is just to move development files to my other computer, no functionality change so far)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5509 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexCell.java, source/de/anomic/index/indexContainerBLOBArray.java, source/de/anomic/index/indexContainerRAMHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRI.java, source/de/anomic/kelondro/kelondroBLOB.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroBLOBCompressor.java, source/de/anomic/kelondro/kelondroBLOBHeapModifier.java, source/de/anomic/kelondro/kelondroBLOBHeapReader.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/plasma/plasmaWordIndex.java
Fri Jan 16 01:02:55 CET 2009
by orbiter
adopted many calls to new api path

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5498 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.html, htroot/WatchWebStructure_p.html, htroot/js/Bookmarks.js, htroot/js/IndexCreate.js, htroot/js/WatchCrawler.js, htroot/rssTerminal.html, htroot/yacy/ui/index.html, htroot/yacy/ui/sidebar/sidebar_1.html, htroot/yacy/ui/sidebar/sidebar_2.html, htroot/yacy/ui/yacyui-bookmarks.html, htroot/yacy/ui/yacyui-search.html, htroot/yacy/ui/yacyui-welcome.html
Fri Jan 16 00:52:58 CET 2009
by orbiter
moved all servlets from htroot/xml to htroot/api
the file server contains a patch that temporary matches all xml paths to api,
that means all interfaces still work. Please adopt all your interfaces to the new path.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5497 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, htroot/WatchCrawler_p.java, htroot/api/blacklists.java, htroot/api/blacklists.xml, htroot/api/blacklists_p.java, htroot/api/blacklists_p.xml, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_bookmarks.json, htroot/api/bookmarks/get_bookmarks.xml, htroot/api/bookmarks/posts/add_p.java, htroot/api/bookmarks/posts/add_p.xml, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/all.xml, htroot/api/bookmarks/posts/delete_p.java, htroot/api/bookmarks/posts/delete_p.xml, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/posts/get.xml, htroot/api/bookmarks/tags/getTag.java, htroot/api/bookmarks/tags/getTag.xml, htroot/api/bookmarks/tags/rename_p.java, htroot/api/bookmarks/tags/rename_p.xml, htroot/api/bookmarks/xbel/xbel.java, htroot/api/bookmarks/xbel/xbel.xml, htroot/api/config_p.java, htroot/api/config_p.xml, htroot/api/feed.java, htroot/api/feed.rss, htroot/api/queues_p.java, htroot/api/queues_p.xml, htroot/api/status_p.java, htroot/api/status_p.xml, htroot/api/util/getpageinfo_p.java, htroot/api/util/getpageinfo_p.xml, htroot/api/util/ynetSearch.java, htroot/api/util/ynetSearch.xml, htroot/api/version.java, htroot/api/version.xml, htroot/api/webstructure.java, htroot/api/webstructure.xml, source/de/anomic/http/httpdFileHandler.java
Tue Jan 13 23:04:38 CET 2009
by orbiter
the Semantic Web moves in!
- added two new api files for document metadata:
- added a XHTML+RDFa html file shows the document metadata in a format that presents the data for rendering and for metadata retrieval. This is a typical document format for a semantic web data structure. the used RDF vocabulary is Dublin Core
- added a xml file that shows the same data as pure DC metadata
- integrated the API into the existing IndexControlURLs interface

With about one billion metadata files (URL metadata) this extension makes the freeworld YaCy network
to one of the probably largest metadata document provider for the semantic web!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5490 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlURLs_p.html, htroot/api/yacydoc.html, htroot/api/yacydoc.java, htroot/api/yacydoc.xml
Mon Jan 12 11:55:48 CET 2009
by orbiter
added an access control to the search interface using white/blacklists:
in the network configuration, you can configure a whiteliste and a blacklist
- blacklistet clients cannot search
- whitelistet client get never any search restrictions
- for all other clients: apply DoS search restrictions
Please see the example configuriation in yacy.network.freeworld.unit
by default, all clients from localhosts get whitlistet.
If you have your own YaCy network, please put all the IPs of your peers into the whitelist

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5475 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.network.freeworld.unit, defaults/yacy.network.intranet.unit, defaults/yacy.network.readme, defaults/yacy.network.webportal.unit, htroot/yacysearch.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/server/serverDomains.java
Tue Jan 06 14:51:59 CET 2009
by orbiter
removed the permanent cache flush and replaced it with a periodic cache flush
The cache is now flushed only for one second every ten seconds. During a crawl the cache
fills up completely, and is only flushed if space is needed for more documents.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5446 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/performance_dht.profile, defaults/yacy.init, htroot/Threaddump_p.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroByteOrder.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/plasma/plasmaWordIndex.java
Tue Jan 06 10:38:08 CET 2009
by orbiter
- refactoring of data access methods to be prepared for new cell data structure
- removed a memory overhead in collections which prevent OOM Exception in low memory configurations

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5443 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/query.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerBLOBHeap.java, source/de/anomic/index/indexContainerRAMHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRI.java, source/de/anomic/kelondro/kelondroAbstractIOChunks.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBLOB.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroBLOBCompressor.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBHeapModifier.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroIOChunks.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaWordIndex.java
Fri Jan 02 13:33:06 CET 2009
by orbiter
removed active profiling in kelondroRowSet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5433 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: lib/svnRevNr/svnRevNrParser.class, source/dbtest.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/kelondro/kelondroAbstractIOChunks.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIOChunks.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java
Fri Jan 02 12:38:20 CET 2009
by orbiter
refactoring (implemented Iterable in kelondroRowCollection)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5432 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverFileUtils.java
Thu Jan 01 23:31:16 CET 2009
by orbiter
replaced the storing procedure for the index ram cache with a method that generates BLOBHeap-compatible dumps
this is a migration step to support a new method to store the web index, which will also based on the same data structure. made also a lot of refactoring for a better structuring of the BLOBHeap class.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5430 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/kelondro/kelondroBLOBGap.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBHeapReader.java, source/de/anomic/kelondro/kelondroBLOBHeapWriter.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java
Mon Dec 29 13:22:13 CET 2008
by orbiter
- added fast fingerprint computation for files (any). Will be used in new index dump method
- refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5415 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAccounts_p.java, htroot/SettingsAck_p.java, htroot/User.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/get.java, htroot/yacy/transfer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/data/userDB.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/index/indexWord.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDigest.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverCodings.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacyURL.java, source/genpw.java, source/migration.java
Fri Dec 19 16:26:01 CET 2008
by orbiter
- added new monitoring servlet at
  http://localhost:8080/PerformanceConcurrency_p.html
- used the new monitoring to do some fine-tuning of the indexing queue

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5402 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceConcurrency_p.html, htroot/PerformanceConcurrency_p.java, htroot/Settings_p.html, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverBlockingThread.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverProcessor.java
Thu Dec 18 01:08:17 CET 2008
by orbiter
- set status of kelondroTree, kelondroBLOBTree and kelondroFlexTable to deprecated
- removed initialization and/or usage of kelondroFlexTable (should meanwhile not be used any more)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5396 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/plasmaRankingCRProcess.java
Wed Dec 17 23:53:06 CET 2008
by orbiter
- refactoring of CrawlEntry and CrawlStacker
- introduced blocking queues in CrawlStacker to make it ready for concurrency
- added a second busy thread for the CrawlStacker
The CrawlStacker is multithreaded. It shall be transformed into a BlockingThread in another step.
The concurrency of the stacker will hopefully solve some problems with cases where DNS blocks.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5395 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/IndexControlRWIs_p.java, htroot/PeerLoadPicture.java, htroot/QuickCrawlLink_p.java, htroot/WatchCrawler_p.java, htroot/rct_p.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/server/serverDomains.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverProcessorJob.java, source/de/anomic/urlRedirector/urlRedirectord.java
Mon Dec 15 01:02:58 CET 2008
by orbiter
- refactoring of CrawlStacker (to prepare it for new multi-Threading to remove DNS lookup bottleneck)
- fix of shallBeOwnWord target computation heuristic


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5392 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAccounts_p.java, htroot/IndexTransfer_p.java, htroot/rct_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferURL.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySeed.java
Wed Dec 10 15:35:01 CET 2008
by orbiter
fine-tuning of cache usage from SVN 5386 and a bug fix for overflow in available() method

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5387 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroBufferedRA.java, source/de/anomic/kelondro/kelondroCachedFileRA.java, source/de/anomic/kelondro/kelondroChannelRA.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroRA.java
Wed Dec 10 12:15:19 CET 2008
by orbiter
- added a read cache to the RAFile interface to RandomAccessFile
- added a write buffer to BLOBHeap
- modified the BLOBBuffer (is now only to buffer non-compressed content)
- added content compression to the HTCache
The new read cache will decrease the start/initialization time of BLOB files,
like the HTCache, RobotsTxt and other BLOBHeap structures.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5386 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroBLOBCompressor.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroRAIOChunks.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/yacy/yacySeedDB.java
Mon Dec 08 01:17:45 CET 2008
by orbiter
fix for problem with userDB and bookmarksDB which was caused by changes in kelondroRA in SVN 5376

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5385 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAccounts_p.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroBLOB.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroBLOBBuffer.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroBufferedRA.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroChannelRA.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroRA.java, source/de/anomic/kelondro/kelondroTree.java
Fri Dec 05 14:55:48 CET 2008
by orbiter
more speed hacks:
- modified and activated write buffer
- increased cache flush factor
- fixed a problem with deadlocking of indexing process

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5382 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Threaddump_p.java, source/de/anomic/index/indexWord.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java
Thu Dec 04 13:54:16 CET 2008
by orbiter
more performance hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5379 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlogComments.java, htroot/IndexControlURLs_p.java, htroot/SettingsAck_p.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/user/sidebar_navigation.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/data/diff.java, source/de/anomic/data/listManager.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/DefinitionListToken.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/data/wiki/tokens/ListToken.java, source/de/anomic/data/wiki/tokens/SimpleToken.java, source/de/anomic/data/wiki/tokens/TableToken.java, source/de/anomic/data/wikiCode.java, source/de/anomic/htmlFilter/htmlFilterAbstractScraper.java, source/de/anomic/htmlFilter/htmlFilterCharacterCoding.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/http/httpdRobotsTxtConfig.java, source/de/anomic/icap/icapHeader.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/index/indexWord.java, source/de/anomic/kelondro/kelondroAbstractIOChunks.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBitfield.java, source/de/anomic/kelondro/kelondroChannelRA.java, source/de/anomic/kelondro/kelondroColumn.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroFullRecords.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroRA.java, source/de/anomic/kelondro/kelondroRAIOChunks.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/parser/ParserInfo.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/xls/xlsParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/server/logging/GuiHandler.java, source/de/anomic/server/logging/serverLog.java, source/de/anomic/server/logging/serverMiniLogFormatter.java, source/de/anomic/server/serverCodings.java, source/de/anomic/server/serverDate.java, source/de/anomic/server/serverMemory.java, source/de/anomic/tools/Punycode.java, source/de/anomic/tools/bitfield.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/xml/crawlHandler.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacyURL.java
Wed Dec 03 16:38:29 CET 2008
by orbiter
performance hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5376 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: bin/searchtest.sh, build.properties, htroot/Blog.java, htroot/Bookmarks.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/Messages_p.java, htroot/News.java, htroot/Threaddump_p.java, htroot/yacysearch.java, htroot/yacysearch.json, htroot/yacysearch.rss, htroot/yacysearchitem.json, htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/URLLicense.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/kelondroAbstractIOChunks.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroBufferedRA.java, source/de/anomic/kelondro/kelondroByteArray.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFullRecords.java, source/de/anomic/kelondro/kelondroIOChunks.java, source/de/anomic/kelondro/kelondroNIOFileRA.java, source/de/anomic/kelondro/kelondroNode.java, source/de/anomic/kelondro/kelondroRA.java, source/de/anomic/kelondro/kelondroRAIOChunks.java, source/de/anomic/server/serverDate.java, source/de/anomic/yacy/yacyURL.java
Wed Dec 03 15:10:01 CET 2008
by lotus
added basic performance page
other performance settings can be found on advanced settings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5375 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, htroot/Performance_p.html, htroot/Performance_p.java, htroot/Settings_p.html, htroot/env/templates/submenuConfig.template, locales/de.lng
Tue Dec 02 16:24:25 CET 2008
by orbiter
added new interactive search feature:
- during the user types search queries, the local database is searched
- results are presented interactively

This was implemented using a new JSON result format for search results in YaCy
- added JSON as file format for servlets
- refactoring of current search servlets (xml and html)
- added JSON output format for search results
- added AJAX-based search page, that uses the yacysearch.json selrvlet to print results as a query is typed

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5373 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/env/templates/header.template, htroot/interactivesearch.html, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearch.json, htroot/yacysearch.rss, htroot/yacysearchitem.html, htroot/yacysearchitem.java, htroot/yacysearchitem.json, htroot/yacysearchitem.xml, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, htroot/yacysearchtrailer.xml, source/de/anomic/data/listManager.java, source/de/anomic/http/httpdFileHandler.java
Mon Nov 24 00:55:08 CET 2008
by orbiter
changed interface to colletctionIndex and adopted all implementing classes:
do not return a result of a double-check when adding entries with addUnique

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5363 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroChunkIterator.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java
Sat Nov 22 00:21:33 CET 2008
by orbiter
fast database initialization and fast start.up of yacy:
- applied knowledge about concurrent files stream reading and index processing from the wikimedia reader
   to the EcoTable initialization process: the file reader is now concurrent to the index generation
- changed also some initialization processes to avoid some pauses during initialization

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5354 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/kelondroChunkIterator.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverCodings.java, source/de/anomic/tools/mediawikiIndex.java, source/yacy.java
Thu Nov 20 19:31:52 CET 2008
by orbiter
- added reader and visualization for mediawiki-export files:
files exported from mediawiki using the xml schema according to
http://www.mediawiki.org/xml/export-0.3/
can be processed to be viewed in a YaCy servlet.
To acces such a file, place it into
DATA/HTCACHE/mediawiki/
i.e. the export from german wikipedia would be:
DATA/HTCACHE/mediawiki/wikipedia.de.xml
This file can then be accessed using the URL
http://localhost:8080/mediawiki_p.html?dump=wikipedia.de.xml&title=YaCy
if this is done the first time, an index file is created
(for this case: more than 4 million lines must be written, this takes about 15 minutes)
Then try the same url again.

- enhanced also the md5 computation speed


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5352 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/mediawiki_p.html, htroot/mediawiki_p.java, source/de/anomic/server/serverCodings.java, source/de/anomic/tools/mediawikiIndex.java, testwiki.xml, testwiki.xml.idx.xml
Fri Nov 14 10:58:56 CET 2008
by orbiter
extended crawling constraints:
- removed never-used secondary crawl depth
- added a must-not-match filter that can be used to exclude urls from a crawl
- added stub for crawl tags which will be used to identify search results that had been produced from specific crawls
please update the yacybar: replace property name 'crawlFilter' with 'mustmatch'.
Additionally, a new parameter named 'mustnotmatch' can be used, which should be by default the empty sring (match-never)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5342 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlProfileEditor_p.html, htroot/CrawlProfileEditor_p.java, htroot/CrawlProfileEditor_p.xml, htroot/CrawlStart_p.html, htroot/CrawlStart_p.java, htroot/QuickCrawlLink_p.java, htroot/WatchCrawler_p.java, htroot/sharedBlacklist_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/urlRedirector/urlRedirectord.java
Wed Nov 12 21:02:19 CET 2008
by orbiter
added linux umlute test files from low012

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5335 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: test/parsertest/umlaute_linux, test/parsertest/umlaute_linux.doc, test/parsertest/umlaute_linux.odp, test/parsertest/umlaute_linux.ods, test/parsertest/umlaute_linux.odt, test/parsertest/umlaute_linux.pdf, test/parsertest/umlaute_linux.ppt, test/parsertest/umlaute_linux.ps, test/parsertest/umlaute_linux.rtf, test/parsertest/umlaute_linux.swf, test/parsertest/umlaute_linux.sxw, test/parsertest/umlaute_linux.txt, test/parsertest/umlaute_linux.xls
Wed Nov 12 01:47:54 CET 2008
by low012
*) changed layout of blacklist adminstration (less cluttered)
*) it is possible to move/edit/delete more than one entry at a time now
*) it is easier to choose a target for blacklist import now
*) fixed several bugs
*) to be continued...

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5330 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.html, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.html, htroot/BlacklistTest_p.java, htroot/Blacklist_p.html, htroot/Blacklist_p.java, htroot/env/base.css, htroot/env/templates/submenuBlacklist.template, htroot/sharedBlacklist_p.html, htroot/sharedBlacklist_p.java
Tue Nov 11 22:33:40 CET 2008
by orbiter
different handling of error cases that occur during loading files with http or ftp:
methods throw exception instead of returning an error string

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5328 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ViewImage.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/plasma/parser/ParserException.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/ymage/ymageOSM.java
Sat Nov 08 21:24:31 CET 2008
by f1ori
* added utf8-encoding to many getBytes-calls
* utf8 should work now


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5323 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/blogBoard.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/yacy/yacyClient.java
Thu Nov 06 11:07:53 CET 2008
by orbiter
- consolidation and refactoring of many DHT target - computing methods
- implemented vertical DHT acceptance ("my own DHT") to accept new targets
- added new target computation for global search: addresses vertical targets also
- enhanced remote crawling: collection of remote crawl urls if queue has less than 100 entries (was: 0 entries)
- better performance value computations for PPM selection in network configuration

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5319 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.init, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.html, htroot/ConfigNetwork_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexTransfer_p.java, htroot/rct_p.java, htroot/yacy/hello.java, htroot/yacy/urls.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyPeerSelection.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Mon Nov 03 01:27:23 CET 2008
by orbiter
Design-check, Extension and Refactoring of DHT target position computation:
- two different computations (but mathematical equivalent) of the DHT distance had been consolidated
- moved from 0.0 .. 1.0 double-range position computation to 0 .. Long.Max range for DHT targets
- added fast Long - to - hash computation
- high-precision target computation of gaps for new peers
- added new target computation for horizontal and vertical DHT targets (not yet in use)
- old horizontal-only DHT targets will be upwards compatible to new horizontal and vertical DHT positions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5318 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/transferRWI.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java
Mon Nov 03 00:30:09 CET 2008
by orbiter
added property index.storeCommons to switch commons storage on or off
with index.storeCommons=false all currently stored commons are deleted!
Default is now 'true', but in future full releases it will be switched to 'false'

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5315 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverFileUtils.java, source/yacy.java
Sat Oct 25 02:11:03 CEST 2008
by low012
*) fixed bug which made it possible to write files outside of the DATA/LIST directory when creating a new blacklist
*) a blacklist will only be created if no blacklist with same name exists (some refactoring has been necessary for this)
*) further minor fixes
*) to be continued...

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5301 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.html, htroot/Blacklist_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/sharedBlacklist_p.java, htroot/xml/blacklists_p.java, source/de/anomic/data/listManager.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/migration.java
Wed Oct 22 20:59:04 CEST 2008
by orbiter
FULL redesign of algorithms in htmlTools to encode/decode strings from/to unicode and html.
The old process used a not really efficient way to detect html encoding strings in texts.
All calling methods had been adoped to call the new class in an enhanced way with less parameters.

Many classes in interfaces used a XML encoding only (instead of full html conversion from unicode to html); this behavior was not changed with this commit but should be controlled again since it points out possible XSS leaks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5295 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/Blog.java, htroot/CrawlProfileEditor_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/PerformanceQueues_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/opensearchdescription.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/xbel/xbel.java, htroot/xml/feed.java, htroot/xml/queues_p.java, htroot/xml/util/getpageinfo_p.java, htroot/yacy/user/ysearch.java, htroot/yacy/user/ysearchitem.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/data/diff.java, source/de/anomic/data/wikiCode.java, source/de/anomic/htmlFilter/htmlFilterCharacterCoding.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/httpd.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/server/serverObjects.java
Wed Oct 22 13:07:14 CEST 2008
by orbiter
added test files for UTF-8 / Umlaute - Testing:
These 3 files contain the same text in different HTML encodings. We use this documents to test if the parser and indexer creates the same set of word hashes for all three texts.

To use these files, run a indexing/crawling on them. To get the files inside the localhost-path, do the following:

cd <yacy-home>
rmdir DATA/HTDOCS/repository
ln -s test/parsertest DATA/HTDOCS/repository

you have then linked the test directory as repository directory which you can reach in yacy if you switch to intranet indexing mode. So the next step is to start yacy, then
- switch to intranet use case
- go to the crawl start page
- the repository directory should be the default path as crawl start
- start the crawl
- search for any word that appears in the demo texts
- search not only for words with umlautel but also for words without umlaute to ensure that you find _all_ three documents
- see how yacy presents the snippet with the text containing umlaute

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5293 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: test/parsertest/umlaute_html.html, test/parsertest/umlaute_iso.html, test/parsertest/umlaute_utf8.html
Mon Oct 20 16:07:09 CEST 2008
by orbiter
performance hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5288 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/tools/gzip.java
Sun Oct 19 20:10:42 CEST 2008
by orbiter
- added some performance tweaks to the new BLOB buffer
- removed the now superfluous HT storage thread
- reduced number of file decompression by shifting the compression moment to the future


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5286 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/performance_dht.profile, defaults/yacy.init, htroot/ProxyIndexingMonitor_p.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroBLOBBuffer.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java
Sat Oct 18 10:57:48 CEST 2008
by orbiter
added another class to handle BLOB writings to the new HTCACHE data storage:
- entries are buffered and written as stream with many entries at once (saves many IO accesses)
- entries are compressed with gzip: increases capacity of cache
- concurrency for stream-writing and compression: all writings to the cache are non-blocking

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5284 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/kelondroBLOB.java, source/de/anomic/kelondro/kelondroBLOBBuffer.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java
Thu Oct 16 23:24:09 CEST 2008
by orbiter
refactoring and new architecture to store the files of the web cache:
- files are not stored any more as individual files
- a new database structure using BLOBHeap files stores many cache entries in common files
- all file-writing procedures had been migrated to generate byte[] objects which are written with the new database methods

this is only an intermediate step to the final architecture, where cached files are written together with their metadata in one single database structure.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5276 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CacheAdmin_p.java, htroot/CrawlResults.java, htroot/IndexCreateIndexingQueue_p.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexDocumentMetadata.java, source/de/anomic/kelondro/kelondroBLOB.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/net/ftpc.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java
Fri Oct 10 10:39:11 CEST 2008
by orbiter
- fix of bug in iterator in kelondroBLOBHeap which caused bug in crawl profile listing
- some refactoring of classes that use kelondroMap (Map instead of HashMap)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5262 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: maptest, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java
Wed Oct 08 20:26:36 CEST 2008
by orbiter
fix for problem with balancer and lost crawl profiles:
if crawl profile ist lost, no robots.txt is loaded any more

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5258 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/urls.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/plasma/plasmaWordIndex.java
Sun Oct 05 22:20:21 CEST 2008
by low012
*) ...and even more languages.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5252 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: langstats/et.lng, langstats/fo.lng, langstats/fy.lng, langstats/gd.lng, langstats/gl.lng, langstats/hr.lng, langstats/hu.lng, langstats/ia.lng, langstats/is.lng, langstats/ku.lng, langstats/la.lng, langstats/lt.lng, langstats/lv.lng, langstats/mi.lng
Sun Oct 05 14:28:40 CEST 2008
by low012
*) more languages

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5250 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: langstats/af.lng, langstats/br.lng, langstats/ca.lng, langstats/co.lng, langstats/cy.lng, langstats/eu.lng, langstats/fi.lng, langstats/no.lng, langstats/ro.lng, langstats/sk.lng, langstats/sl.lng, langstats/sv.lng, langstats/sw.lng
Fri Oct 03 20:57:02 CEST 2008
by orbiter
- fixed several bugs in database management functions
- fixed a display bug for the performance graph
- fixed deadlock when initialization of awt happens simultanously
- removed some debugging output

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5245 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/xml/queues_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroStack.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaProfiling.java
Wed Sep 24 23:09:12 CEST 2008
by apfelmaennchen
YaCy-UI:
- added bookmark folder widget to sidebar (it uses the xbel.xml with XSLT!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5206 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/jquery.treeview.css, htroot/yacy/ui/img/treeview/file.gif, htroot/yacy/ui/img/treeview/folder-closed.gif, htroot/yacy/ui/img/treeview/folder.gif, htroot/yacy/ui/img/treeview/minus.gif, htroot/yacy/ui/img/treeview/plus.gif, htroot/yacy/ui/img/treeview/treeview-black-line.gif, htroot/yacy/ui/img/treeview/treeview-black.gif, htroot/yacy/ui/img/treeview/treeview-default-line.gif, htroot/yacy/ui/img/treeview/treeview-default.gif, htroot/yacy/ui/img/treeview/treeview-famfamfam-line.gif, htroot/yacy/ui/img/treeview/treeview-famfamfam.gif, htroot/yacy/ui/img/treeview/treeview-gray-line.gif, htroot/yacy/ui/img/treeview/treeview-gray.gif, htroot/yacy/ui/img/treeview/treeview-red-line.gif, htroot/yacy/ui/img/treeview/treeview-red.gif, htroot/yacy/ui/index.html, htroot/yacy/ui/js/jquery.treeview.min.js, htroot/yacy/ui/js/jquery.xslTransform.js, htroot/yacy/ui/js/sarissa.js, htroot/yacy/ui/js/sarissa_ieemu_xpath.js, htroot/yacy/ui/sidebar/sidebar_2.html, htroot/yacy/ui/xsl/xbel2folder-ul.xsl, htroot/yacy/ui/yacyui-bookmarks.html, htroot/yacy/ui/yacyui-search.html, htroot/yacy/ui/yacyui-welcome.html
Wed Sep 24 01:30:25 CEST 2008
by orbiter
fix for wrong index initialization after network switch

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5203 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/Banner.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/CrawlStart_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/Status.java, htroot/yacysearch.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNetwork.java
Sun Sep 21 02:04:42 CEST 2008
by orbiter
added ranking and evaluation of language type in a search
the wanted language is taken from the browser user-agent string

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5192 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/yacy/search.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/tools/iso639.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java
Sat Sep 20 00:19:11 CEST 2008
by orbiter
- added language detection using metadata from documents: html and odt documents provide this information
- metadata and results from statistical analysis are compared and result is printed out as debug lines
- added ranking profile for wanted language
- added class with ISO 639 table, a list of all valid country codes that will be used for the language identification

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5187 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Ranking_p.java, htroot/xml/util/getpageinfo_p.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/plasma/parser/doc/docParser.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/rtf/rtfParser.java, source/de/anomic/plasma/parser/sevenzip/sevenzipParser.java, source/de/anomic/plasma/parser/swf/swfParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/xls/xlsParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/tools/iso639.java, source/de/anomic/yacy/yacyURL.java
Fri Sep 19 13:45:11 CEST 2008
by lotus
replaced some put(String, String) by putHTML(String, String) on serverObjects respond
in htroot/ root
didn't touch htroot/xml/
this should solve potential xss issues

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5184 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigUpdate_p.java, htroot/CookieTest_p.java, htroot/FeedReader_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Network.java, htroot/News.java, htroot/ProxyIndexingMonitor_p.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/User.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchWebStructure_p.java, htroot/Wiki.java, htroot/index.java, htroot/sharedBlacklist_p.java, htroot/yacysearchitem.java
Tue Sep 16 23:56:23 CEST 2008
by orbiter
fixed bug with wrong proxy result enqueueing. See:
http://forum.yacy-websuche.de/viewtopic.php?p=8130#p8130
- removed the online status property. This influenced the proxy behavior and created some complexity that was not needed because the online status was never used as it was ceated for (offline browsing)
- checked all proxy identification procedures during crawling and enhanced transparency and error checking
- fixed a proxy identification routine that caused the wrong selection of the proxy result queue

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5173 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/Network.html, htroot/Network.java, htroot/SettingsAck_p.java, htroot/Status.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/yacy/yacyCore.java
Mon Sep 15 19:18:07 CEST 2008
by apfelmaennchen
Major update to YaCy-UI

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5162 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/base.css, htroot/yacy/ui/css/jquery.flexigrid.css, htroot/yacy/ui/css/jquery.tabs.css, htroot/yacy/ui/img/add.png, htroot/yacy/ui/img/close.png, htroot/yacy/ui/img/flexigrid/._fhbg.gif, htroot/yacy/ui/img/flexigrid/._wbg.gif, htroot/yacy/ui/img/flexigrid/bg.gif, htroot/yacy/ui/img/flexigrid/btn-sprite.gif, htroot/yacy/ui/img/flexigrid/ddn.png, htroot/yacy/ui/img/flexigrid/dn.png, htroot/yacy/ui/img/flexigrid/fhbg.gif, htroot/yacy/ui/img/flexigrid/first.gif, htroot/yacy/ui/img/flexigrid/hl.png, htroot/yacy/ui/img/flexigrid/last.gif, htroot/yacy/ui/img/flexigrid/line.gif, htroot/yacy/ui/img/flexigrid/load.gif, htroot/yacy/ui/img/flexigrid/load.png, htroot/yacy/ui/img/flexigrid/magnifier.png, htroot/yacy/ui/img/flexigrid/next.gif, htroot/yacy/ui/img/flexigrid/prev.gif, htroot/yacy/ui/img/flexigrid/up.png, htroot/yacy/ui/img/flexigrid/uup.png, htroot/yacy/ui/img/flexigrid/wbg.gif, htroot/yacy/ui/index.html, htroot/yacy/ui/js/jquery-flexigrid.js, htroot/yacy/ui/js/jquery-keynav.js, htroot/yacy/ui/sidebar/sidebar_1.html, htroot/yacy/ui/sidebar/sidebar_2.html, htroot/yacy/ui/yacyui-admin.html, htroot/yacy/ui/yacyui-bookmarks.html, htroot/yacy/ui/yacyui-classic.html, htroot/yacy/ui/yacyui-ranking.html, htroot/yacy/ui/yacyui-search.html, htroot/yacy/ui/yacyui-welcome.html, htroot/yacy/ui/yacyui-widget.html
Mon Sep 15 11:17:05 CEST 2008
by orbiter
- renamed Comparison to compare_yacy
- added more search engines
- some refactoring and added a list that is used to present the search engine list in a specific order
- added simpleheader and no-header options
- added the compare search to the simple header
- added default compare search page selection storage - after re-start you get the same default search engines as you selected before

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5157 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.init, htroot/compare_yacy.html, htroot/compare_yacy.java, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template
Fri Sep 12 13:51:48 CEST 2008
by orbiter
shifted index deletion-on-exit rule to the class where the errors are produced

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5141 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexCollectionRI.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBufferedEcoFS.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexWidthArray.java, source/de/anomic/kelondro/kelondroIOChunks.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroRA.java, source/de/anomic/kelondro/kelondroRAIOChunks.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacySearch.java
Thu Sep 04 21:41:57 CEST 2008
by orbiter
- added domain statistic generation to IndexControlURLs_p.html servlet
- added 'delete all' button to all results of such a domain statistic output which causes that all urls to this domain are deleted
- extended stack cleaner to clean also the statistics: they are not completely destroyed, only the smallest counting domains are removed


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5117 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlResults.html, htroot/CrawlResults.java, htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyURL.java
Wed Sep 03 02:30:21 CEST 2008
by orbiter
added logging conditions to all fine and finest log line calls
this will prevent an overhead for the generation of the log lines in case that they then are not printed

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5102 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/rct_p.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/translator.java, source/de/anomic/http/HttpConnectionInfo.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/plasma/parser/sevenzip/SZParserExtractCallback.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java
Wed Sep 03 01:49:48 CEST 2008
by orbiter
- fixed problem with searching with quotes (still not complete, but not as bad as before)
- fixed parsing of crawl-delay statements when seconds were given with float numbers
- enhanced performance of profiling (not too many loggings; not more than one per second)
- removed some debug output
- fixed wrong return type in logging
- added a logging condition in httpd to prevent that logging statements are generated when they are not written (should be added everywhere!)
- fixed wrong word distance computation in RWI management


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5101 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacysearch.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/robotsParser.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpd.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/server/logging/serverLog.java, source/de/anomic/server/serverProfiling.java
Mon Aug 25 20:11:47 CEST 2008
by orbiter
modifications towards a single database operation to read/write http header and cached file at once:
- removed distinction between header file types for http and ftp; ftp is simulated by using http properties
- removed all old resourceInfo classes that handled this distinction
- introduced a new distinction between http request and http response objects
- unified new response objects with two other object types that had been introduced elsewhere
- changed all servlet call methods to use the new http request header object type
- divided static object keys for http header properties into request and response types
- refactoring here and there (a large number of type changes and many methods merged/moved)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5079 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/CacheResource_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/FeedReader_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/imagetest.java, htroot/index.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/xml/blacklists_p.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/delete_p.java, htroot/xml/bookmarks/posts/get.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/tags/rename_p.java, htroot/xml/bookmarks/xbel/xbel.java, htroot/xml/config_p.java, htroot/xml/feed.java, htroot/xml/queues_p.java, htroot/xml/status_p.java, htroot/xml/util/getpageinfo_p.java, htroot/xml/util/ynetSearch.java, htroot/xml/version.java, htroot/xml/webstructure.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacy/user/sidebar_history.java, htroot/yacy/user/sidebar_navigation.java, htroot/yacy/user/ysearch.java, htroot/yacy/user/ysearchitem.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/userDB.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterInputStream.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/JakartaCommonsHttpResponse.java, source/de/anomic/http/httpChunkedInputStream.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRequestHeader.java, source/de/anomic/http/httpResponseHeader.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverDate.java, source/de/anomic/server/servletProperties.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/ymageOSM.java, source/yacy.java
Wed Aug 20 09:54:56 CEST 2008
by danielr
- changed default browser from netscape to firefox
- fixed "Inefficient use of keySet iterator instead of entrySet iterator" [WMI_WRONG_MAP_ITERATOR, FindBugs]
- fixed some possible null pointer accesses


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5063 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/BlacklistCleaner_p.java, htroot/IndexShare_p.java, htroot/News.java, htroot/Ranking_p.java, htroot/Threaddump_p.java, htroot/yacy/hello.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/userDB.java, source/de/anomic/http/httpHeader.java, source/de/anomic/index/indexDefaultReferenceBlacklist.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverSystem.java, source/de/anomic/yacy/yacyPeerActions.java, source/yacy.java
Tue Aug 19 16:10:40 CEST 2008
by orbiter
Preparations to replace the HTCache with a new storage data structure:
- refactoring of the HTCache (separation of cache entry)
- added new storage class for BLOBs. (not used yet, this is half-way to a new structure)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5062 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/ViewFile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/http/httpdProxyCacheEntry.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/kelondro/kelondroBLOBArray.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/ymage/ymageOSM.java
Sun Aug 17 12:16:32 CEST 2008
by danielr
- fixed "yacy2yacy no proxy"-problem


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5058 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpRemoteProxyConfig.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/server/serverSystem.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Wed Aug 13 12:37:53 CEST 2008
by orbiter
- extended new performance graph (better timing)
- added paths for new libraries in classpath for eclipse
- refactoring to remove compiler warnings (static access to finals variables)
- removed some unused import

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5055 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, source/de/anomic/htmlFilter/htmlFilterInputStream.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/httpd.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/tools/cryptbig.java
Sun Aug 10 17:26:13 CEST 2008
by f1ori
* add build target "deb" to create debian packages from svn
  still needs testing...


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5051 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: addon/yacyInit.m4, build.xml, debian/changelog, debian/compat, debian/control, debian/copyright, debian/postinst, debian/postrm, debian/preinst, debian/prerm, debian/rules
Sat Aug 09 14:00:31 CEST 2008
by danielr
- fixed UnsupportedEncoding (in proxy) using defaultCharset if no characterEncoding can be determined
- serverFileUtils.copy* use now Charset instead of String
- added some warnings for ignored exceptions


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5043 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/CacheAdmin_p.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/server/serverFileUtils.java
Fri Aug 08 03:40:28 CEST 2008
by danielr
replaced httpd.parseMultipart() by a 'right' implementation


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5040 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: NOTICE, build.xml, lib/commons-fileupload-1.2.1.License, lib/commons-fileupload-1.2.1.jar, lib/commons-io-1.4.License, lib/commons-io-1.4.jar, lib/servlet-api.License, lib/servlet-api.jar, source/de/anomic/http/httpd.java
Wed Aug 06 21:43:12 CEST 2008
by danielr
* removed some warnings of findbugs (http://findbugs.sf.net)
- removed unnecessary code (unused variables, String.toString)
- corrected some calculations (cast int to double or long ;)
- improved little performance (using Integer.valueOf() instead of new Integer)
- log if some File-actions fail (mkdir(), delete(), ...) and some ignored exceptions
- finalized some (more) fields
- finally close some streams
- made inner classes static if not using environment
- generalized some equals (from specificClass to Object)
- fixed some potential nullpointer accesses


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5039 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/BlacklistCleaner_p.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/CookieTest_p.java, htroot/IndexImport_p.java, htroot/Settings_ProxyAccess.inc, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewImage.java, htroot/ViewProfile.java, htroot/WebStructurePicture_p.java, htroot/xml/bookmarks/posts/get.java, htroot/xml/bookmarks/xbel/xbel.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/ui/ymarks.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiCode.java, source/de/anomic/htmlFilter/htmlFilterAbstractScraper.java, source/de/anomic/htmlFilter/htmlFilterAbstractTransformer.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/htmlFilter/htmlFilterInputStream.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/EasySSLProtocolSocketFactory.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/JakartaCommonsHttpResponse.java, source/de/anomic/http/httpChunkedInputStream.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdByteCountInputStream.java, source/de/anomic/http/httpdByteCountOutputStream.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexPhrase.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/index/indexWord.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBitfield.java, source/de/anomic/kelondro/kelondroBufferedEcoFS.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroByteArray.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroColumn.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFlexWidthArray.java, source/de/anomic/kelondro/kelondroHandle.java, source/de/anomic/kelondro/kelondroHashtable.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroMHashMap.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/kelondro/kelondroObjectSpace.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSortStack.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/kelondro/kelondroXMLTables.java, source/de/anomic/language/identification/LanguageFilenameFilter.java, source/de/anomic/language/identification/LanguageStatistics.java, source/de/anomic/language/identification/LanguageStatisticsHolder.java, source/de/anomic/net/ftpc.java, source/de/anomic/net/whois.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/sevenzip/ByteArrayIInStream.java, source/de/anomic/plasma/parser/swf/swfParser.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaStore.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/server/logging/logParsers/LogParserPLASMA.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverByteBuffer.java, source/de/anomic/server/serverCharBuffer.java, source/de/anomic/server/serverCodings.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverDate.java, source/de/anomic/server/serverDomains.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/server/serverSystem.java, source/de/anomic/tools/bbCode.java, source/de/anomic/tools/bitfield.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyTray.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/AnimGifEncoder.java, source/de/anomic/ymage/ymageChart.java, source/de/anomic/ymage/ymageFontGenerator.java, source/de/anomic/ymage/ymageGraph.java, source/de/anomic/ymage/ymageICOParser.java, source/de/anomic/ymage/ymageMatrix.java, source/migration.java, source/yacy.java
Sat Aug 02 15:57:00 CEST 2008
by danielr
* refactoring
- moved constants from plasmaSwitchboard to own class (all 232 ;)
- moved remoteProxy-Methods to httpRemoteProxyConfig, better names
- removed some unnecessary code (else-statements)
* formatting (correct indentation)
* minor bugfixes (due to findbugs.sf.net)
* hopefully fixed "missing quote" (announcing StringParts as UTF-8)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5031 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CacheAdmin_p.java, htroot/CacheResource_p.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/ConfigRobotsTxt_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/IndexShare_p.java, htroot/PeerLoadPicture.java, htroot/PerformanceQueues_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/WatchCrawler_p.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/index.java, htroot/robots.java, htroot/xml/queues_p.java, htroot/xml/status_p.java, htroot/yacy/list.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/data/wiki/tokens/SimpleToken.java, source/de/anomic/data/wiki/tokens/TableToken.java, source/de/anomic/htmlFilter/htmlFilterAbstractScraper.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterImageEntry.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/DefaultCharsetFilePart.java, source/de/anomic/http/DefaultCharsetStringPart.java, source/de/anomic/http/EasySSLProtocolSocketFactory.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/JakartaCommonsHttpResponse.java, source/de/anomic/http/httpRemoteProxyConfig.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/http/httpdRobotsTxtConfig.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroNIOFileRA.java, source/de/anomic/kelondro/kelondroObjectSpace.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRelations.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroStack.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/kelondro/kelondroXMLTables.java, source/de/anomic/language/identification/LanguageFilenameFilter.java, source/de/anomic/language/identification/LanguageStatistics.java, source/de/anomic/net/ftpc.java, source/de/anomic/plasma/cache/ftp/ResourceInfo.java, source/de/anomic/plasma/cache/http/ResourceInfo.java, source/de/anomic/plasma/parser/sevenzip/sevenzipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardConstants.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/logging/LogalizerHandler.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverCachedFileOutputStream.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverDate.java, source/de/anomic/server/serverDomains.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverSystem.java, source/de/anomic/server/servletProperties.java, source/de/anomic/tools/Punycode.java, source/de/anomic/tools/consoleInterface.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/tools/nxTools.java, source/de/anomic/tools/tarTools.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/xml/RSSFeed.java, source/de/anomic/xml/RSSMessage.java, source/de/anomic/xml/crawlHandler.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacyTray.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/ymageBMPParser.java, source/migration.java, source/yacy.java
Sat Aug 02 14:12:04 CEST 2008
by danielr
added final where possible

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5030 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/CacheResource_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStart_p.java, htroot/FeedReader_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/imagetest.java, htroot/index.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/xml/blacklists_p.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/delete_p.java, htroot/xml/bookmarks/posts/get.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/tags/rename_p.java, htroot/xml/bookmarks/xbel/xbel.java, htroot/xml/config_p.java, htroot/xml/feed.java, htroot/xml/queues_p.java, htroot/xml/status_p.java, htroot/xml/util/getpageinfo_p.java, htroot/xml/util/ynetSearch.java, htroot/xml/version.java, htroot/xml/webstructure.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/ymarks.java, htroot/yacy/urls.java, htroot/yacy/user/sidebar_history.java, htroot/yacy/user/sidebar_navigation.java, htroot/yacy/user/ysearch.java, htroot/yacy/user/ysearchitem.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/dbtest.java, source/de/anomic/crawler/AbstractImporter.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/ImporterException.java, source/de/anomic/crawler/ImporterManager.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/ResourceObserver.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/robotsParser.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/URLLicense.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/htmlTools.java, source/de/anomic/data/listManager.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/abstractWikiParser.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/AbstractToken.java, source/de/anomic/data/wiki/tokens/DefinitionListToken.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/data/wiki/tokens/ListToken.java, source/de/anomic/data/wiki/tokens/SimpleToken.java, source/de/anomic/data/wiki/tokens/TableToken.java, source/de/anomic/data/wiki/wikiParserException.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/data/wikiCode.java, source/de/anomic/htmlFilter/htmlFilterAbstractScraper.java, source/de/anomic/htmlFilter/htmlFilterAbstractTransformer.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/htmlFilter/htmlFilterImageEntry.java, source/de/anomic/htmlFilter/htmlFilterInputStream.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/EasySSLProtocolSocketFactory.java, source/de/anomic/http/EasyX509TrustManager.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/HttpConnectionInfo.java, source/de/anomic/http/httpChunkedInputStream.java, source/de/anomic/http/httpChunkedOutputStream.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRemoteProxyConfig.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdBoundedSizeOutputStream.java, source/de/anomic/http/httpdByteCountInputStream.java, source/de/anomic/http/httpdByteCountOutputStream.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdLimitExceededException.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/http/httpdRobotsTxtConfig.java, source/de/anomic/icap/icapHeader.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexAbstractReferenceBlacklist.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/index/indexDefaultReferenceBlacklist.java, source/de/anomic/index/indexPhrase.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/index/indexReferenceBlacklist.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/index/indexWord.java, source/de/anomic/kelondro/kelondroAbstractIOChunks.java, source/de/anomic/kelondro/kelondroAbstractOrder.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBinSearch.java, source/de/anomic/kelondro/kelondroBitfield.java, source/de/anomic/kelondro/kelondroBufferedEcoFS.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroBufferedRA.java, source/de/anomic/kelondro/kelondroByteArray.java, source/de/anomic/kelondro/kelondroByteOrder.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroCloneableMapIterator.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroColumn.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroException.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroFixedWidthArray.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFlexWidthArray.java, source/de/anomic/kelondro/kelondroFullRecords.java, source/de/anomic/kelondro/kelondroHandle.java, source/de/anomic/kelondro/kelondroHashtable.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroLock.java, source/de/anomic/kelondro/kelondroMHashMap.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroMicroDate.java, source/de/anomic/kelondro/kelondroNIOFileRA.java, source/de/anomic/kelondro/kelondroNaturalOrder.java, source/de/anomic/kelondro/kelondroObjectBuffer.java, source/de/anomic/kelondro/kelondroObjectSpace.java, source/de/anomic/kelondro/kelondroOutOfLimitsException.java, source/de/anomic/kelondro/kelondroProfile.java, source/de/anomic/kelondro/kelondroRAIOChunks.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRelations.java, source/de/anomic/kelondro/kelondroRotateIterator.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSortStack.java, source/de/anomic/kelondro/kelondroSortStore.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroStack.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/kelondro/kelondroXMLTables.java, source/de/anomic/language/identification/Identificator.java, source/de/anomic/language/identification/LanguageFilenameFilter.java, source/de/anomic/language/identification/LanguageStatistics.java, source/de/anomic/language/identification/LanguageStatisticsHolder.java, source/de/anomic/net/ftpc.java, source/de/anomic/net/natLib.java, source/de/anomic/net/whois.java, source/de/anomic/plasma/cache/ResourceInfoFactory.java, source/de/anomic/plasma/cache/UnsupportedProtocolException.java, source/de/anomic/plasma/cache/ftp/ResourceInfo.java, source/de/anomic/plasma/cache/http/ResourceInfo.java, source/de/anomic/plasma/parser/AbstractParser.java, source/de/anomic/plasma/parser/ParserException.java, source/de/anomic/plasma/parser/ParserInfo.java, source/de/anomic/plasma/parser/bzip/bzipParser.java, source/de/anomic/plasma/parser/doc/docParser.java, source/de/anomic/plasma/parser/gzip/gzipParser.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/plasma/parser/mimeType/rssDetector.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/rtf/rtfParser.java, source/de/anomic/plasma/parser/sevenzip/ByteArrayIInStream.java, source/de/anomic/plasma/parser/sevenzip/SZParserExtractCallback.java, source/de/anomic/plasma/parser/sevenzip/sevenzipParser.java, source/de/anomic/plasma/parser/swf/swfParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/xls/xlsParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaStore.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/logging/ConsoleOutErrHandler.java, source/de/anomic/server/logging/ConsoleOutHandler.java, source/de/anomic/server/logging/GuiHandler.java, source/de/anomic/server/logging/LogalizerHandler.java, source/de/anomic/server/logging/logParsers/LogParserPLASMA.java, source/de/anomic/server/logging/serverLog.java, source/de/anomic/server/logging/serverMiniLogFormatter.java, source/de/anomic/server/logging/serverSimpleLogFormatter.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverAbstractThread.java, source/de/anomic/server/serverByteBuffer.java, source/de/anomic/server/serverCachedFileOutputStream.java, source/de/anomic/server/serverCharBuffer.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverCodings.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverCoreSocket.java, source/de/anomic/server/serverDate.java, source/de/anomic/server/serverDomains.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/server/serverMemory.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverPlainSwitch.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverProcessorJob.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/server/serverSemaphore.java, source/de/anomic/server/serverSwitchAbstractAction.java, source/de/anomic/server/serverSystem.java, source/de/anomic/server/servletProperties.java, source/de/anomic/tools/ListDirs.java, source/de/anomic/tools/PKCS12Tool.java, source/de/anomic/tools/Punycode.java, source/de/anomic/tools/bbCode.java, source/de/anomic/tools/bitfield.java, source/de/anomic/tools/consoleInterface.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/tools/disorderHeap.java, source/de/anomic/tools/disorderSet.java, source/de/anomic/tools/enumerateFiles.java, source/de/anomic/tools/gzip.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/tools/md5DirFileFilter.java, source/de/anomic/tools/nxTools.java, source/de/anomic/tools/tarTools.java, source/de/anomic/tools/yFormatter.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/xml/RSSFeed.java, source/de/anomic/xml/RSSMessage.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/xml/crawlHandler.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFile.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFtp.java, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.java, source/de/anomic/yacy/yacyAccessible.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyTray.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/AnimGifEncoder.java, source/de/anomic/ymage/ymage8BitFontGenerator.java, source/de/anomic/ymage/ymageBMPParser.java, source/de/anomic/ymage/ymageCaptcha.java, source/de/anomic/ymage/ymageChart.java, source/de/anomic/ymage/ymageDemoApplet.java, source/de/anomic/ymage/ymageFontGenerator.java, source/de/anomic/ymage/ymageGraph.java, source/de/anomic/ymage/ymageICOParser.java, source/de/anomic/ymage/ymageImageParser.java, source/de/anomic/ymage/ymageMatrix.java, source/de/anomic/ymage/ymageOSM.java, source/de/anomic/ymage/ymageToolCircle.java, source/de/anomic/ymage/ymageToolPrint.java, source/genpw.java, source/migration.java, source/screenshot.java, source/yacy.java
Sun Jul 20 19:14:51 CEST 2008
by orbiter
- removed superfluous copyright statement
- updated my email address

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5011 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/CacheResource_p.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/IndexCleaner_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/imagetest.java, htroot/index.java, htroot/opensearchdescription.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/delete_p.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/tags/rename_p.java, htroot/xml/queues_p.java, htroot/xml/util/getpageinfo_p.java, htroot/xml/webstructure.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/ymarks.java, htroot/yacysearch.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ErrorURL.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/htmlTools.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/translator.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/AbstractToken.java, source/de/anomic/data/wiki/tokens/DefinitionListToken.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/data/wiki/tokens/ListToken.java, source/de/anomic/data/wiki/tokens/SimpleToken.java, source/de/anomic/data/wiki/tokens/TableToken.java, source/de/anomic/data/wiki/tokens/Token.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/htmlFilter/htmlFilterAbstractScraper.java, source/de/anomic/htmlFilter/htmlFilterAbstractTransformer.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/htmlFilter/htmlFilterImageEntry.java, source/de/anomic/htmlFilter/htmlFilterInputStream.java, source/de/anomic/htmlFilter/htmlFilterScraper.java, source/de/anomic/htmlFilter/htmlFilterTransformer.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/httpChunkedInputStream.java, source/de/anomic/http/httpChunkedOutputStream.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRemoteProxyConfig.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdBoundedSizeOutputStream.java, source/de/anomic/http/httpdByteCountInputStream.java, source/de/anomic/http/httpdByteCountOutputStream.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdLimitExceededException.java, source/de/anomic/http/httpdRobotsTxtConfig.java, source/de/anomic/icap/icapHeader.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexRI.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/kelondro/kelondroAbstractIOChunks.java, source/de/anomic/kelondro/kelondroAbstractOrder.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroArray.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBinSearch.java, source/de/anomic/kelondro/kelondroBitfield.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroBufferedRA.java, source/de/anomic/kelondro/kelondroByteArray.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroCloneableIterator.java, source/de/anomic/kelondro/kelondroCoding.java, source/de/anomic/kelondro/kelondroColumn.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroFixedWidthArray.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFlexWidthArray.java, source/de/anomic/kelondro/kelondroHashtable.java, source/de/anomic/kelondro/kelondroIOChunks.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroLock.java, source/de/anomic/kelondro/kelondroMHashMap.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroNIOFileRA.java, source/de/anomic/kelondro/kelondroNaturalOrder.java, source/de/anomic/kelondro/kelondroObjectBuffer.java, source/de/anomic/kelondro/kelondroObjectSpace.java, source/de/anomic/kelondro/kelondroOrder.java, source/de/anomic/kelondro/kelondroOutOfLimitsException.java, source/de/anomic/kelondro/kelondroProfile.java, source/de/anomic/kelondro/kelondroRA.java, source/de/anomic/kelondro/kelondroRAIOChunks.java, source/de/anomic/kelondro/kelondroRotateIterator.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroStack.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/kelondro/kelondroXMLTables.java, source/de/anomic/language/identification/Identificator.java, source/de/anomic/language/identification/LanguageFilenameFilter.java, source/de/anomic/language/identification/LanguageStatistics.java, source/de/anomic/language/identification/LanguageStatisticsHolder.java, source/de/anomic/net/natLib.java, source/de/anomic/net/whois.java, source/de/anomic/plasma/cache/IResourceInfo.java, source/de/anomic/plasma/cache/ResourceInfoFactory.java, source/de/anomic/plasma/cache/ftp/ResourceInfo.java, source/de/anomic/plasma/cache/http/ResourceInfo.java, source/de/anomic/plasma/parser/AbstractParser.java, source/de/anomic/plasma/parser/Parser.java, source/de/anomic/plasma/parser/ParserException.java, source/de/anomic/plasma/parser/bzip/bzipParser.java, source/de/anomic/plasma/parser/doc/docParser.java, source/de/anomic/plasma/parser/gzip/gzipParser.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/plasma/parser/mimeType/rssDetector.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/rtf/rtfParser.java, source/de/anomic/plasma/parser/sevenzip/ByteArrayIInStream.java, source/de/anomic/plasma/parser/sevenzip/SZParserExtractCallback.java, source/de/anomic/plasma/parser/sevenzip/sevenzipParser.java, source/de/anomic/plasma/parser/swf/swfParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/xls/xlsParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaStore.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/logging/ConsoleOutErrHandler.java, source/de/anomic/server/logging/ConsoleOutHandler.java, source/de/anomic/server/logging/GuiHandler.java, source/de/anomic/server/logging/LogalizerHandler.java, source/de/anomic/server/logging/logParsers/LogParser.java, source/de/anomic/server/logging/logParsers/LogParserPLASMA.java, source/de/anomic/server/logging/serverLog.java, source/de/anomic/server/logging/serverMiniLogFormatter.java, source/de/anomic/server/logging/serverSimpleLogFormatter.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverByteBuffer.java, source/de/anomic/server/serverCachedFileOutputStream.java, source/de/anomic/server/serverCharBuffer.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverCodings.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverCoreSocket.java, source/de/anomic/server/serverDate.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/server/serverHandler.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverPlainSwitch.java, source/de/anomic/server/serverSemaphore.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/server/serverSwitchAbstractAction.java, source/de/anomic/server/serverSwitchAction.java, source/de/anomic/server/serverSystem.java, source/de/anomic/tools/PKCS12Tool.java, source/de/anomic/tools/bbCode.java, source/de/anomic/tools/bitfield.java, source/de/anomic/tools/consoleInterface.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/disorderHeap.java, source/de/anomic/tools/disorderSet.java, source/de/anomic/tools/enumerateFiles.java, source/de/anomic/tools/gzip.java, source/de/anomic/tools/loaderCore.java, source/de/anomic/tools/loaderProcess.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/tools/md5DirFileFilter.java, source/de/anomic/tools/nxTools.java, source/de/anomic/tools/yFormatter.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFile.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFtp.java, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.java, source/de/anomic/yacy/yacyAccessible.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacySeedUploader.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/ymage8BitFontGenerator.java, source/de/anomic/ymage/ymageCaptcha.java, source/de/anomic/ymage/ymageChart.java, source/de/anomic/ymage/ymageFontGenerator.java, source/genpw.java, source/migration.java, source/yacy.java
Sun Jul 20 02:21:07 CEST 2008
by orbiter
removed superfluous copyright statement

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5010 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/CacheResource_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest_p.java, htroot/IndexCleaner_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/Steering.java, htroot/Threaddump_p.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/imagetest.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/delete_p.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/tags/rename_p.java, htroot/xml/queues_p.java, htroot/xml/status_p.java, htroot/xml/util/getpageinfo_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/ymarks.java, htroot/yacysearch.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/robotsParser.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/htmlTools.java, source/de/anomic/data/listManager.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/AbstractToken.java, source/de/anomic/data/wiki/tokens/DefinitionListToken.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/data/wiki/tokens/ListToken.java, source/de/anomic/data/wiki/tokens/SimpleToken.java, source/de/anomic/data/wiki/tokens/TableToken.java, source/de/anomic/data/wiki/tokens/Token.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/data/wikiCode.java
Sun Jul 13 09:51:45 CEST 2008
by lotus
new yacyTray
this will make a YaCy icon in the tray area on supported platforms
enabled by default
the search page will open on double click

used JDIC 0.9.4 from https://jdic.dev.java.net/

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4992 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, addon/YaCy_TrayIcon.gif, build.xml, defaults/yacy.init, htroot/Status.java, htroot/Status_p.inc, lib/jdic-0.9.4.License, lib/jdic.jar, lib/linux/jdic_stub.jar, lib/linux/x86/libjdic.so, lib/linux/x86/libtray.so, lib/sunos/jdic_stub.jar, lib/sunos/sparc/libjdic.so, lib/sunos/sparc/libtray.so, lib/sunos/x86/libjdic.so, lib/sunos/x86/libtray.so, lib/windows/jdic_stub.jar, lib/windows/x86/jdic.dll, lib/windows/x86/tray.dll, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyTray.java, yacy.logging
Fri Jul 11 09:34:48 CEST 2008
by orbiter
discovered and removed a (possibly large) memory leak:
many classes used the kelondroMapDataMining (was: kelondroMapObjects) which adds statistical
functions to the kelondroMap (was: kelondroObjects), but these functions were not used by these
classes. Especially the HTCACHE and robots.txt database allocate a very large number of objects
for statistical use, but never used them. By replacing the kelondroMapDataMining with the
kelondroMap object for these classes now less memory is allocated.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4986 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/plasma/plasmaHTCache.java
Fri Jul 11 09:15:46 CEST 2008
by orbiter
refactoring of method calling for objects from kelondroMapDataMining

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4985 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/yacy/yacySeedDB.java
Fri Jul 11 00:08:16 CEST 2008
by orbiter
refactoring of kelondroObjects (mainly renaming to kelondroMap)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4982 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/kelondro/kelondroMap.java, source/de/anomic/kelondro/kelondroMapDataMining.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Thu Jul 10 21:57:09 CEST 2008
by orbiter
created new main menu entry 'Customization and Integration'
- moved some already existing servlets to this menu
- renamed the skin servlet to appearance
- added a set-to-default-button to the search page appearance setting
- removed the peer profile servlet which is now replaced by a field in the new appearance servlet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4980 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAppearance_p.html, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.html, htroot/ConfigProfile_p.html, htroot/ConfigRobotsTxt_p.html, htroot/env/templates/header.template, htroot/env/templates/submenuConfig.template, htroot/env/templates/submenuCustomizationIntegration.template, htroot/env/templates/submenuPublication.template, source/de/anomic/data/bookmarksDB.java
Thu Jul 10 02:47:37 CEST 2008
by orbiter
Major update to BLOB data structures:
- introduced a new BLOB file format: kelondroBLOBHeap. This is a flat file with an index in RAM.
  very similar to the eco-tables, but with flexible value sizes. It will replace the kelondroBLOBTree,
  which is based on a kelondroTree, a file-AVL-based index data structure.
- the HTCACHE header file was replaced by the new blob heap file structure
- the robots.txt file was replaced by the new blob heap file structure
- the robots parser was enhanced (bugfixing for double-loading of the same robots.txt)
- other BLOB-dependent data structures were prepared to use also the new BLOB heap
- fixed a bug in the snippet fetch process: the file header was not written to the header index
There should now be less IO during snippet fetch and during crawling


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4978 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Wiki.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/robotsParser.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/kelondro/kelondroBLOB.java, source/de/anomic/kelondro/kelondroBLOBHeap.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMapObjects.java, source/de/anomic/kelondro/kelondroObjects.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java
Sat Jul 05 02:35:20 CEST 2008
by orbiter
- refactoring of robots parser
- no more keep-order parameter in remove (it was not possible to make this strict, and not useful)
- some small enhancements in balancer
- robots parser without references in switchboard
- changes synchronization in robots

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4969 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/Network.java, htroot/xml/util/getpageinfo_p.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/robotsParser.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRelations.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyNewsDB.java
Fri Jul 04 13:03:03 CEST 2008
by orbiter
added many missing user-agent declarations for yacy http client connections.
the most important fix was the addition of the yacybot user-agent for robots.txt loading,
because web masters look for that access to see if the crawler behaves correctly.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4968 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigLanguage_p.java, htroot/ConfigSkins_p.java, htroot/CookieTest_p.html, htroot/CookieTest_p.java, htroot/Network.java, htroot/sharedBlacklist_p.java, htroot/xml/util/getpageinfo_p.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/robotsParser.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/net/natLib.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Fri Jul 04 01:37:04 CEST 2008
by orbiter
added options to configure the 'corporate identity'-icons, the home page link and the greeting line from
the skin menue. Additionally an example is given there how to integrate a search page with an iframe.
Please see the skin menu.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4967 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigSkins_p.html, htroot/ConfigSkins_p.java, htroot/env/base.css, htroot/index.html, htroot/index.java, htroot/yacysearch.html, htroot/yacysearch.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/kelondro/kelondroMicroDate.java, source/de/anomic/kelondro/kelondroRelations.java
Thu Jul 03 15:08:37 CEST 2008
by orbiter
- modified and enhanced the crawl balancer: better list export, fixing of damaged crawl queue at start-up, re-sorting at start-up to enhance domain order
- added option to set minimum crawl delta for domains in balancer
- added default values to crawl deltas in yacy.init
- added configuration for these deltas in performance queues
- enhanced performance setting computation (more time for indexing queue for a faster flush
- remote crawling is now enabled during local crawling if indexer has space and time for more links
- added database stub for new distributed file system
- refactoring of time computation to get an abstraction level that will be used by a TTL rule in new distributed file system

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4966 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigBasic.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, htroot/xml/queues_p.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/kelondro/kelondroMicroDate.java, source/de/anomic/kelondro/kelondroRelations.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java
Mon Jun 30 23:47:53 CEST 2008
by orbiter
modified the diskUsage class regarding the following two aspects:
1. The usage and dependency of the plasmaSwitchboad was used many times in the past but this was
a bad mistake. The classes should be independent from the switchboard to support a better abstraction. Therefore the object was removed. The parameters from the switchboard are computed outside and then handed over.
2. the class is considered as a tightly connected to hardware resources. Classes which handle data that cannot be replicated because it would need to replicate hadware should not support dynamic object allocation, but should be coded as collection of private static methods. Therefore all class objects had been transformed into static private objects.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4961 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/tools/consoleInterface.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/yacy/resourceObserver.java, source/de/anomic/yacy/yacyDHTAction.java
Sun Jun 29 20:35:15 CEST 2008
by danielr
- set staticIP at startUp
- added setting for reduced menu (simpleMenu)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4959 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigBasic.html, htroot/ConfigBasic.java, htroot/PerformanceQueues_p.html, htroot/SettingsAck_p.java, htroot/env/templates/header.template, htroot/env/templates/submenuConfig.template, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/servletProperties.java, source/de/anomic/yacy/yacyCore.java
Wed Jun 18 01:56:39 CEST 2008
by orbiter
some code-cleanup and possible speed enhancements in different core methods

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4935 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaWordIndex.java
Sat Jun 14 12:24:58 CEST 2008
by danielr
- WatchCrawler shows status without JavaScript
- Performance can be scaled + DHT-profile
- names for pool-threads
- some small refactorings


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4923 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/performance_dht.profile, htroot/ConfigBasic.java, htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, htroot/WatchCrawler_p.html, htroot/WatchCrawler_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/server/NamePrefixThreadFactory.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/yacy/yacyAccessible.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Sun Jun 08 01:12:24 CEST 2008
by orbiter
refactoring:
find a better way to store BLOBs; generalize current BLOG data structure (kelondroDyn)
and prepare it to replace it with something better. The best candidate is the kelondroHeap,
which will become the kelondroBLOBHeap;
removed also some never-used classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4902 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/kelondro/kelondroBLOB.java, source/de/anomic/kelondro/kelondroBLOBTree.java, source/de/anomic/kelondro/kelondroMapObjects.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/kelondro/kelondroObjects.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Fri Jun 06 18:01:27 CEST 2008
by danielr
- organize imports
- removed potential null pointer accesses
- removed unnecessary casts


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4893 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/Collage.java, htroot/ConfigAccounts_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigSkins_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/FeedReader_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Statistics.java, htroot/Status.java, htroot/Threaddump_p.java, htroot/User.java, htroot/ViewFile.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/Wiki.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/xml/blacklists_p.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/get.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/tags/rename_p.java, htroot/xml/bookmarks/xbel/xbel.java, htroot/xml/config_p.java, htroot/xml/queues_p.java, htroot/xml/util/getpageinfo_p.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/ymarks.java, htroot/yacy/user/sidebar_history.java, htroot/yacy/user/sidebar_navigation.java, htroot/yacy/user/ysearch.java, htroot/yacy/user/ysearchitem.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/dbtest.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/ImporterManager.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/robotsParser.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/data/wiki/tokens/ListToken.java, source/de/anomic/data/wiki/tokens/SimpleToken.java, source/de/anomic/data/wiki/tokens/TableToken.java, source/de/anomic/htmlFilter/htmlFilterAbstractScraper.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/htmlFilter/htmlFilterImageEntry.java, source/de/anomic/http/EasySSLProtocolSocketFactory.java, source/de/anomic/http/EasyX509TrustManager.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdByteCountInputStream.java, source/de/anomic/http/httpdByteCountOutputStream.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapHeader.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexAbstractReferenceBlacklist.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexDefaultReferenceBlacklist.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexReferenceBlacklist.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/kelondro/kelondroAbstractIOChunks.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroDynTree.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFullRecords.java, source/de/anomic/kelondro/kelondroHandle.java, source/de/anomic/kelondro/kelondroHashtable.java, source/de/anomic/kelondro/kelondroHeap.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMapObjects.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroNIOFileRA.java, source/de/anomic/kelondro/kelondroNaturalOrder.java, source/de/anomic/kelondro/kelondroObjectSpace.java, source/de/anomic/kelondro/kelondroObjects.java, source/de/anomic/kelondro/kelondroRotateIterator.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/kelondro/kelondroXMLTables.java, source/de/anomic/language/identification/Identificator.java, source/de/anomic/language/identification/LanguageStatistics.java, source/de/anomic/net/ftpc.java, source/de/anomic/plasma/cache/ftp/ResourceInfo.java, source/de/anomic/plasma/cache/http/ResourceInfo.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordConnotation.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/logging/ConsoleOutErrHandler.java, source/de/anomic/server/logging/GuiHandler.java, source/de/anomic/server/logging/LogalizerHandler.java, source/de/anomic/server/logging/logParsers/LogParserPLASMA.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverByteBuffer.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverDomains.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/server/serverSystem.java, source/de/anomic/tools/PKCS12Tool.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/diskUsage.java, source/de/anomic/tools/disorderSet.java, source/de/anomic/tools/enumerateFiles.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/tools/nxTools.java, source/de/anomic/xml/crawlHandler.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/resourceObserver.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFtp.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/AnimGifEncoder.java, source/de/anomic/ymage/ymageBMPParser.java, source/de/anomic/ymage/ymageICOParser.java, source/de/anomic/ymage/ymageMatrix.java, source/de/anomic/ymage/ymageToolPrint.java, source/yacy.java
Thu Jun 05 13:01:20 CEST 2008
by orbiter
- enhanced handling of own IP address inside seed
- prevention of false information of own IP address
- enabled searching before an own IP address is assigned (before first ping happened)
- removed warning about limited search function
- added better time-out settings for peer-ping process (10 seconds complete, 5 seconds for back-ping)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4883 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/Network.java, htroot/SettingsAck_p.java, htroot/Status.html, htroot/sharedBlacklist_p.java, htroot/yacy/hello.java, htroot/yacysearch.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java
Thu Jun 05 00:24:00 CEST 2008
by orbiter
- fixed several NPE bugs
- fixed loosing of own seed hash (hopefully)
- fixed a bug with crawl start s beginning with (bookmark) files
- added better IP recognition during hello process


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4882 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlStart_p.html, htroot/WatchCrawler_p.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/search.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java
Wed Jun 04 23:34:57 CEST 2008
by orbiter
- fixed at least two causes for a NPE after a use case switch.
   A large refactoring was neccessary
- added another crawl start option: automatic restriction to sub-path
- removed crawlStartSimple and renamed crawl start expert
   to crawl start (without expert)
- some changes to texts in crawl start
- added some more deletions when an web index is deleted:
   delete also queues and robots cache


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4881 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlStart_p.html, htroot/CrawlStart_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexTransfer_p.java, htroot/MessageSend_p.java, htroot/Network.java, htroot/Status.java, htroot/WatchCrawler_p.html, htroot/WatchCrawler_p.java, htroot/env/templates/submenuIndexCreate.template, htroot/rct_p.java, htroot/www/welcome.java, htroot/xml/status_p.java, htroot/yacy/hello.java, htroot/yacy/search.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java
Wed Jun 04 01:06:01 CEST 2008
by orbiter
added superfast search result computation which can be obtained for local search when snippet fetching is disabled. An example search for the rss interface would be:
http://localhost:8080/yacysearch.rss?query=yacy&Enter=Search&contentdom=text&count=10&resource=local&verify=false
(just add "&verify=false")

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4878 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Status.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java
Sun May 25 20:35:38 CEST 2008
by orbiter
- refactoring of seedURL handling
- additional check for seedURL pointing to localhost: deny such peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4851 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.init, htroot/Network.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Status.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/search.java, source/de/anomic/http/httpd.java, source/de/anomic/icap/icapd.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacyURL.java
Sat May 24 23:30:22 CEST 2008
by orbiter
- fix for bad reset of index / bad index location after deletion
- some modification of rssTerminal window location and size

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4850 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Status.html, htroot/terminal_p.html, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroNIOFileRA.java, source/de/anomic/kelondro/kelondroRA.java
Sat May 24 14:30:50 CEST 2008
by orbiter
added a deletion button to indexControlRWIs that deletes the complete web index

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4847 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRI.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMapObjects.java, source/de/anomic/kelondro/kelondroObjects.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/language/identification/LanguageStatisticsHolder.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacySeedDB.java
Wed May 21 17:23:56 CEST 2008
by lotus
* automatic update for windows
pleas disable before release because 2nd update fails at the moment
and commandline handling has to be improved for windows
* update via new unTar class
please review stream- and exceptionhandling because I'm fairly new to Java
maybe it can be done concurrent
* updated windows startscripts to values from yacy.init

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4832 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigUpdate_p.html, htroot/ConfigUpdate_p.java, htroot/Steering.java, htroot/yacysearchitem.java, source/de/anomic/server/serverSystem.java, source/de/anomic/tools/tarTools.java, source/de/anomic/yacy/yacyVersion.java, startYACY.bat, startYACY_noconsole.bat
Tue May 20 11:29:01 CEST 2008
by orbiter
fixed several details about network switching, default password, random password and localhost authentification

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4830 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigAccounts_p.html, htroot/ConfigAccounts_p.java, htroot/ConfigNetwork_p.html, htroot/ConfigNetwork_p.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/language/identification/Identificator.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverAbstractSwitch.java, source/yacy.java
Sun May 18 23:24:05 CEST 2008
by low012
*) first version of a way to determine the language a text is written in (not perfect, but it works)
*) statistical data of languages can be found in the *.lng files in the new directory called "langfiles"

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4824 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: langstats/da.lng, langstats/de.lng, langstats/en.lng, langstats/fr.lng, langstats/nl.lng, source/de/anomic/language/identification/Identificator.java, source/de/anomic/language/identification/LanguageFilenameFilter.java, source/de/anomic/language/identification/LanguageStatistics.java
Thu May 15 13:26:43 CEST 2008
by orbiter
implemented switch for admin authorization from localhost:
- access is granted for localhost users to administration pages by default
- the default setting can be changed in the BasicConfig.html page
- if the BasicConfig page was accessed with post and no password was submitted, a random password is generated
- a headless installation MUST give a password upon first call of the configuration page, otherwise they will not be able to access it again
- if no password is given within 10 minutes after start-up, a random password is generated

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4804 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigBasic.html, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.html, htroot/ConfigNetwork_p.java, htroot/env/templates/header.template, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/plasmaSwitchboard.java
Wed May 14 23:36:02 CEST 2008
by orbiter
- added option to switch between yacy networks, especially between the two default networks (freeworld and intranet),
  from the ConfigNetwork online interface
- to make this possible, a large refactoring and reorganisation of data structures was necessary

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4803 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.networks, htroot/AccessTracker_p.java, htroot/Banner.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.html, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigRobotsTxt_p.java, htroot/Connections_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStartSimple_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/Wiki.java, htroot/YaCySearchPluginFF.java, htroot/index.java, htroot/opensearchdescription.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/xml/queues_p.java, htroot/xml/status_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/result.java, htroot/yacy/ui/ymarks.java, htroot/yacy/urls.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/IndexingStack.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/wiki/abstractWikiParser.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyPeerActions.java
Wed May 14 22:30:44 CEST 2008
by orbiter
- changed seed file storage to clear text
- fixed kill script
- fixed saving of seed file (had been corrupted by latest changes)
- some refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4799 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, killYACY.sh, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java
Tue May 13 13:46:20 CEST 2008
by orbiter
- added RWI transmissions to the event terminal
- fixed bug in Collage
- added 'embedded mode' to collage
- integrated Collage to terminal_p as iframe in embedded mode (Pictures now visible in terminal_p)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4797 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Collage.html, htroot/Collage.java, htroot/Status.html, htroot/terminal_p.html, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/xml/RSSFeed.java, source/de/anomic/yacy/yacyPeerActions.java
Tue May 13 00:23:29 CEST 2008
by orbiter
- better domain graph, faster when more links exist, looks better
- new authorization rule: localhost is always authorized for administration. This solves many problems with ajax, and also fixed a problem in rssTerminal
- fix bug in RSSFeed which prevented that entries had been recognized as individual, new entries
- added reloading/updating of status image on status page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4796 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Status.html, htroot/processing/domaingraph/applet/domaingraph.jar, htroot/processing/domaingraph/applet/domaingraph.java, htroot/processing/domaingraph/applet/domaingraph.pde, htroot/processing/domaingraph/domaingraph.pde, htroot/rssTerminal.html, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/xml/RSSFeed.java, source/de/anomic/xml/RSSMessage.java
Thu May 08 00:49:07 CEST 2008
by orbiter
?added new terminal servlet using current visualization methods and a new one: a processing (processing.org) applet.
the new servlet can be found at
http://localhost:8080/terminal_p.html
..to be enhanced..

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4773 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.xml, htroot/env/terminal.css, htroot/processing/domaingraph/applet/domaingraph.jar, htroot/processing/domaingraph/applet/domaingraph.java, htroot/processing/domaingraph/applet/domaingraph.pde, htroot/processing/domaingraph/applet/index.html, htroot/processing/domaingraph/applet/loading.gif, htroot/processing/domaingraph/domaingraph.pde, htroot/processing/put_in_libraries/animation.zip, htroot/processing/put_in_libraries/physics.zip, htroot/terminal_p.html
Wed May 07 01:05:48 CEST 2008
by orbiter
enhanced rssTerminal
- 3 lines possible
- distinguishing of private and public data, if not authorized only public data is shown
- shows now more events, including local searches in clear text if user is logged in
- simplyfied peer events
- better recognition of 'real' new peers
- presentation of peer pings from other peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4771 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlStartSimple_p.java, htroot/IndexTransfer_p.java, htroot/Status.html, htroot/rct_p.java, htroot/rssTerminal.html, htroot/xml/feed.java, htroot/yacy/search.java, htroot/yacy/ui/result.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/xml/RSSFeed.java, source/de/anomic/xml/RSSMessage.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java
Tue May 06 15:44:38 CEST 2008
by orbiter
refactoring:
moved importer classes to crawler and plasma package

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4770 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexImport_p.html, htroot/IndexImport_p.java, htroot/WatchCrawler_p.java, source/de/anomic/crawler/AbstractImporter.java, source/de/anomic/crawler/Importer.java, source/de/anomic/crawler/ImporterException.java, source/de/anomic/crawler/ImporterManager.java, source/de/anomic/crawler/NoticeURLImporter.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/plasma/plasmaDbImporter.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/yacy.java
Tue May 06 02:32:41 CEST 2008
by orbiter
refactoring: moved all crawler classes into their own package

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4768 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.xml, htroot/Collage.java, htroot/CrawlProfileEditor_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/QuickCrawlLink_p.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/xml/queues_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/urls.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlEntry.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/ErrorURL.java, source/de/anomic/crawler/FTPLoader.java, source/de/anomic/crawler/HTTPLoader.java, source/de/anomic/crawler/LoaderMessage.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/crawler/ProtocolLoader.java, source/de/anomic/crawler/ResultImages.java, source/de/anomic/crawler/ResultURLs.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/robotsParser.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/plasma/dbImport/SitemapImporter.java, source/de/anomic/plasma/dbImport/plasmaCrawlNURLImporter.java, source/de/anomic/plasma/parser/ParserException.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java
Tue May 06 01:13:47 CEST 2008
by orbiter
major step forward to network switching (target is easy switch to intranet or other networks .. and back)
This change is inspired by the need to see a network connected to the index it creates in a indexing team.
It is not possible to divide the network and the index. Therefore all control files for the network was moved to the network within the INDEX/<network-name> subfolder.
The remaining YACYDB is superfluous and can be deleted.
The yacyDB and yacyNews data structures are now part of plasmaWordIndex. Therefore all methods, using static access to yacySeedDB had to be rewritten. A special problem had been all the port forwarding methods which had been tightly mixed with seed construction. It was not possible to move the port forwarding functions to the place, meaning and usage of plasmaWordIndex. Therefore the port forwarding had been deleted (I guess nobody used it and it can be simulated by methods outside of YaCy).
The mySeed.txt is automatically moved to the current network position. A new effect causes that every network will create a different local seed file, which is ok, since the seed identifies the peer only against the network (it is the purpose of the seed hash to give a peer a location within the DHT).
No other functional change has been made. The next steps to enable network switcing are:
- shift of crawler tables from PLASMADB into the network (crawls are also network-specific)
- possibly shift of plasmaWordIndex code into yacy package (index management is network-specific)
- servlet to switch networks 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4765 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, defaults/yacy.init, htroot/AccessTracker_p.java, htroot/Banner.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigRobotsTxt_p.java, htroot/Connections_p.java, htroot/CrawlResults.java, htroot/CrawlStartSimple_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexTransfer_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/QuickCrawlLink_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.html, htroot/SettingsAck_p.java, htroot/Settings_p.html, htroot/Settings_p.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/Wiki.java, htroot/YaCySearchPluginFF.java, htroot/index.java, htroot/opensearchdescription.java, htroot/rct_p.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/xml/queues_p.java, htroot/xml/status_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/result.java, htroot/yacy/ui/ymarks.java, htroot/yacy/urls.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/wiki/abstractWikiParser.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdAlternativeDomainNames.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/crawler/plasmaFTPLoader.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/plasmaCrawlEntry.java, source/de/anomic/plasma/plasmaCrawlResultURLs.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverDomains.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyURL.java, source/migration.java, source/yacy.java
Sat May 03 11:06:00 CEST 2008
by danielr
refactoring (initialized static fields, removed empty if/else, serialized some fields in serializable classes)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4755 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/IndexCleaner_p.java, htroot/LogStatistics_p.java, htroot/News.java, htroot/SettingsAck_p.java, htroot/Threaddump_p.java, htroot/xml/bookmarks/xbel/xbel.java, htroot/yacy/ui/ymarks.java, source/de/anomic/data/listManager.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/httpdByteCountOutputStream.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/kelondroHandle.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/net/ftpc.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/logging/ConsoleOutHandler.java, source/de/anomic/server/portForwarding/sch/serverPortForwardingSch.java, source/de/anomic/server/serverCachedFileOutputStream.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverCoreSocket.java, source/de/anomic/server/serverDate.java, source/de/anomic/server/serverMemory.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/server/serverSystem.java, source/de/anomic/tools/disorderHeap.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Fri May 02 00:40:42 CEST 2008
by orbiter
refactoring as result of
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=959&p=7560#p7560

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4752 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyURL.java
Wed Apr 30 00:31:05 CEST 2008
by orbiter
- modified, simplified old kelondroHTCache object; I believe it should be replaced by something completely new
- removed tree data type in kelondroHTCache
- added new class kelondroHeap; may be the core for a storage object that will once replace the many-files strategy of kelondroHTCache
- removed compatibility mode in indexRAMRI


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4747 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/kelondro/kelondroHeap.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java
Sat Apr 26 03:00:10 CEST 2008
by orbiter
added new news window into the status page. At this moment it is just a test.
The news inside the window are about peer arrivals and departures, remote search accesses and crawls

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4739 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Status.html, htroot/js/query.js, htroot/rssTerminal.html, htroot/xml/feed.java, htroot/yacy/search.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverCore.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyVersion.java
Thu Apr 24 23:31:07 CEST 2008
by orbiter
refactoring of RSS reader

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4736 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: bin/searchprofiling, htroot/FeedReader_p.java, htroot/rct_p.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/xml/RSSFeed.java, source/de/anomic/xml/RSSMessage.java, source/de/anomic/xml/RSSReader.java, source/de/anomic/xml/rssReader.java, source/de/anomic/yacy/yacyClient.java
Thu Apr 24 15:31:55 CEST 2008
by orbiter
small corrections and enhancements after search timing profiling
search should be a little bit faster now

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4734 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterInputStream.java, source/de/anomic/http/httpHeader.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaWordIndex.java
Thu Apr 24 10:42:08 CEST 2008
by orbiter
added new default profiles to distinguish snippet fetch for local and global search
the difference is, that a local search will no not cause a re-indexing of loaded pages

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4731 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/WatchWebStructure_p.java, htroot/yacy/ui/ymarks.java, htroot/yacy/user/sidebar_history.java, htroot/yacy/user/sidebar_navigation.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/ymageOSM.java
Tue Apr 22 00:42:49 CEST 2008
by orbiter
- refactoring of Colage back-end: move to plasma package
- renamed also the plasmaCrawlResults to have a consistent naming for url and image queues
- added a double-check for the images
- added additional queues for the images: all worse-quality images go there, so the queue can be used also if no sizes are given; no image is lost
- added a cleanup for the stacks so they cannot flood the memory

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4722 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Collage.html, htroot/Collage.java, htroot/xml/util/getpageinfo_p.java, source/de/anomic/plasma/plasmaCrawlResultImages.java, source/de/anomic/plasma/plasmaCrawlResultURLs.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java
Sun Apr 20 23:36:25 CEST 2008
by orbiter
- better logging when rejecting a url because it is not in declared domain
- more XSS attack protection

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4720 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Config_p.java, htroot/Connections_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/rct_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferURL.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaCrawlEURL.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyClient.java
Sat Apr 19 13:31:41 CEST 2008
by apfelmaennchen
- major rework on YaCy-UI
- search results are retrieved from rss/xml, no other servlet needed
- added double accordion sidebar menus

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4718 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/base.css, htroot/yacy/ui/css/jquery.pagination.css, htroot/yacy/ui/css/jquery.tablesorter.pager.css, htroot/yacy/ui/index.html, htroot/yacy/ui/js/jquery-faviconize-1.0.js, htroot/yacy/ui/js/jquery.accordion.js, htroot/yacy/ui/js/keynav.js, htroot/yacy/ui/ranking.html, htroot/yacy/ui/sidebar/sidebar.html, htroot/yacy/ui/sidebar/sidebar_admin.html, htroot/yacy/ui/welcome.html, htroot/yacy/ui/ymarks.html, htroot/yacy/ui/ysearch.html
Fri Apr 18 15:58:56 CEST 2008
by orbiter
- added servlet to configure the search page greeting line
- added information output about the current network definition in the network servlet
- better description and usage of profile entries in User Profile servlet regarding FOAF format
- reformatting of menues at status page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4710 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigNetwork_p.html, htroot/ConfigNetwork_p.java, htroot/ConfigPeerProfile_p.html, htroot/ConfigPeerProfile_p.java, htroot/ConfigProfile_p.html, htroot/env/templates/submenuConfig.template, source/de/anomic/server/serverFileUtils.java
Sat Apr 12 13:39:48 CEST 2008
by danielr
- disabled redirects in proxy (so client sees real path)
- added connection stats (only connections currently in use)
- remove "old" connections (closed or idle for some time)
- synchronized shared parts of proxyHandler


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4682 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Connections_p.html, htroot/Connections_p.java, source/de/anomic/http/HttpConnectionInfo.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/JakartaCommonsHttpResponse.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverFileUtils.java
Fri Apr 11 00:47:05 CEST 2008
by orbiter
refactoring of HttpClient Writer processes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4678 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/robotsParser.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/JakartaCommonsHttpResponse.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Thu Apr 10 15:24:55 CEST 2008
by orbiter
- enhanced performance of Eco-Tables: less time-consuming size() - operations
- will increase speed of indexing and collection.index creation


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4675 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/kelondro/kelondroBufferedEcoFS.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverCore.java
Thu Apr 10 11:53:07 CEST 2008
by orbiter
- refacotring of stream copy handling to support time-consuming operations
- made usage of BufferedStreams explizit to distinct different copy method in serverFileUtils (byte-by-byte and using an own buffer)
- introduced another timeout setting (java internal property)
- more restrictions to clients accessing a single host (a security setting to prevent DoS by mistake)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4674 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/robotsParser.java, source/de/anomic/http/HttpResponse.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Thu Apr 10 01:35:20 CEST 2008
by orbiter
- removed class with only one static method
- removed connection method with too long time-out

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4672 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/robotsParser.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Tue Apr 08 23:36:33 CEST 2008
by orbiter
- some refactoring (sorry Daniel, hab in deinem Code rumgewütet)
- fixed broken downloads (flush was missing)
- different problem handling when download is corrupted
- different default values in yacy.init

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4669 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/ConfigUpdate_p.java, source/de/anomic/http/HttpResponse.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/JakartaCommonsHttpResponse.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexContainerHeap.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/yacy/yacyVersion.java
Tue Apr 08 23:17:40 CEST 2008
by danielr
- fixed POST in proxy
- prepared http connection tracking
- refactoring (mainly moving StreamTools to serverFileUtils)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4668 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Connections_p.html, htroot/Connections_p.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/HttpConnectionInfo.java, source/de/anomic/http/HttpResponse.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/yacy/yacyClient.java
Tue Apr 08 16:44:39 CEST 2008
by orbiter
- enhanced performance graph (more info)
- added conditions for rarely used logging lines to prevent unnecessary CPU usage for non-printed info

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4667 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceGraph.java, source/dbtest.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/dbImport/dbImportManager.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaDHTFlush.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/portForwarding/sch/serverPortForwardingSch.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverMemory.java, source/de/anomic/ymage/ymageChart.java
Tue Apr 08 13:55:59 CEST 2008
by orbiter
fix for http://forum.yacy-websuche.de/viewtopic.php?p=6806#p6806
- removed all InputStream.available() because this does not work for files > 2GB
- iterator terminate when a IOException occurs
- added handling of non-executing index.add methods to enhance assert usage
- added index for file indexes > 2GB, to be used in new indexHeap

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4666 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexContainerHeap.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroBytesLongMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/server/serverCore.java, yacy.logging
Sat Apr 05 15:17:16 CEST 2008
by danielr
replaced httpc with Apache Jakarta Commons HttpClient (includes some refactoring ;)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4640 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, build.xml, defaults/yacy.init, htroot/Blog.java, htroot/Bookmarks.java, htroot/ConfigLanguage_p.java, htroot/ConfigSkins_p.java, htroot/Connections_p.html, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/SettingsAck_p.java, htroot/ViewFile.java, htroot/sharedBlacklist_p.java, htroot/xml/util/getpageinfo_p.java, htroot/yacy/ui/ymarks.java, libx/commons-httpclient-3.1.jar, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/listManager.java, source/de/anomic/data/robotsParser.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/HttpClient.java, source/de/anomic/http/HttpFactory.java, source/de/anomic/http/HttpResponse.java, source/de/anomic/http/JakartaCommonsHttpClient.java, source/de/anomic/http/JakartaCommonsHttpResponse.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRemoteProxyConfig.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/kelondro/kelondroBufferedRA.java, source/de/anomic/net/natLib.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaStore.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverCore.java, source/de/anomic/tools/StreamTools.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/tools/nxTools.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/de/anomic/ymage/ymageCaptcha.java, source/yacy.java
Wed Apr 02 15:18:23 CEST 2008
by orbiter
strong speed enhancements for the index cache dump and restore:
storage and loading is 30 times faster! a cache of 100000 RWIs needed 180 seconds
to store and 100 seconds to restore; now the same cache needs only 6 seconds to store and
3 seconds to restore. The cache size has decreased now by 30% (95 MB instead of 150 MB).

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4634 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexContainerHeap.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRIReader.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroNaturalOrder.java, source/de/anomic/kelondro/kelondroOrder.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/plasma/plasmaWordIndex.java
Mon Mar 31 17:41:19 CEST 2008
by orbiter
speed enhancements for crawler and url retrieval (affects also search speed)
- concurrency for LURL-fetching: this can be done using a concurrent lookup into the separated url databases. Concurrency is possible because there is no IO during lookup. The more LURL-Tables are present, the better is the speedup. More CPUs will increase speed
- because a large number of LURL-lookups are made during crawling (for double-check), the LURL-Lookup speed enhancements enhances also crawling speed
- search speed also profits from LURL-lookup enhancement
- changed some flushing parameters in word index caching which should make better use of large word index caches and should speed up indexing
- removed flush chunksize parameter, because this was only useful for IO path enhancement feature which was removed some weeks ago to prevent blocking and deadlocks during search requests

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4628 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, defaults/yacy.init, htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/plasmaCrawlNURL.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverCore.java
Sun Mar 30 01:03:44 CET 2008
by orbiter
enhanced the concurrency handling of indexing process (better queue size control, better data concept, better shutdown behavior)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4617 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/plasma/plasmaCrawlNURL.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverBlockingThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverProcessorJob.java
Fri Mar 28 12:56:28 CET 2008
by orbiter
added concurrency to indexing process:
- the methods {parsing, semantic analysis (condensing), structure analysis (web structure)} in the serialized indexing path had been made concurrent.
- four BlockingQueues handle concurrency and hand-over of the indexing objects, the last object in the queue is stored into a blockingQueue of maximum size 1 to serialize the process for storage (which uses IO and therefore here should not be deserialized)
- a concurrency of (CPUs + 1) is default. Single-CPU users will profil from the change because large files cannot block the indexing process any more.
- removed the secondary indexing thread, which is superfluous now. Concurrency is default for all users.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4609 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: defaults/yacy.init, htroot/SettingsAck_p.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverBusyThread.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverProcessorJob.java
Thu Mar 27 13:03:16 CET 2008
by orbiter
- refactoring of serverThreads: renaming to distinguish busy-threads and blocking-threads
- added blockingThreads which are threads that are not driven by pause times but by BlockingQueue lookup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4606 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/PeerLoadPicture.java, htroot/PerformanceQueues_p.java, htroot/SettingsAck_p.java, source/dbtest.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/portForwarding/sch/serverPortForwardingSch.java, source/de/anomic/server/serverAbstractBlockingThread.java, source/de/anomic/server/serverAbstractBusyThread.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverAbstractThread.java, source/de/anomic/server/serverBlockingThread.java, source/de/anomic/server/serverBusyThread.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverInstantBlockingThread.java, source/de/anomic/server/serverInstantBusyThread.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/server/serverThread.java
Wed Mar 26 23:43:38 CET 2008
by orbiter
- preparation of parsing/indexing queue for concurrent execution
- remote crawl receipts are now transmitted concurrently in separate threads (makes remove crawls much faster!)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4605 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/IndexCreateIndexingQueue_p.java, htroot/Status.java, htroot/xml/queues_p.java, source/de/anomic/kelondro/kelondroBufferedEcoFS.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/plasma/plasmaWordIndex.java
Wed Mar 26 16:37:49 CET 2008
by orbiter
- refactoring and abstraction of index reference (urls) handling: blacklisting is part of reference filtering
- refactoring of word/phrase handling: word abstraction from condenser becomes part of index element handling
- removed unused code parts from condenser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4603 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/sharedBlacklist_p.java, htroot/xml/blacklists_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/result.java, htroot/yacy/ui/ymarks.java, htroot/yacy/urls.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/listManager.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/index/indexAbstractReferenceBlacklist.java, source/de/anomic/index/indexDefaultReferenceBlacklist.java, source/de/anomic/index/indexPhrase.java, source/de/anomic/index/indexReferenceBlacklist.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/index/indexURLReference.java, source/de/anomic/index/indexWord.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaCrawlResults.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/yacy.java
Wed Mar 26 15:13:05 CET 2008
by orbiter
- separated the LURL data storage and Crawl result stack for process supervision.
  this is another step to enable multiple, concurrent fulltext-indexes
- another try to make the yacy-httpc more stable

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4602 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/xml/queues_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/result.java, htroot/yacy/ui/ymarks.java, htroot/yacy/urls.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/http/httpc.java, source/de/anomic/index/indexRepositoryReference.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/plasmaCrawlResults.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/yacy.java
Sun Mar 23 01:55:04 CET 2008
by orbiter
- removed some debugging code from search process - should speed up now
- added some profiling code to search event - more time details in PerformanceSearch_p.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4594 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigNetwork_p.java, htroot/PerformanceSearch_p.html, htroot/PerformanceSearch_p.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java
Sat Mar 22 02:28:37 CET 2008
by orbiter
refactoring of switchboard queueing

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4591 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/CacheResource_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSkins_p.java, htroot/ConfigUpdate_p.java, htroot/Config_p.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStartExpert_p.java, htroot/CrawlStartSimple_p.java, htroot/FeedReader_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/PeerLoadPicture.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SearchEventPicture.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Statistics.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/Threaddump_p.java, htroot/User.java, htroot/User_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/WikiHelp.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/imagetest.java, htroot/index.java, htroot/opensearchdescription.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/xml/blacklists_p.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/delete_p.java, htroot/xml/bookmarks/posts/get.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/tags/rename_p.java, htroot/xml/bookmarks/xbel/xbel.java, htroot/xml/config_p.java, htroot/xml/queues_p.java, htroot/xml/status_p.java, htroot/xml/util/getpageinfo_p.java, htroot/xml/version.java, htroot/xml/webstructure.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/result.java, htroot/yacy/ui/ritem.java, htroot/yacy/ui/ymarks.java, htroot/yacy/urls.java, htroot/yacy/user/sidebar_history.java, htroot/yacy/user/sidebar_navigation.java, htroot/yacy/user/ysearch.java, htroot/yacy/user/ysearchitem.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/data/translator.java, source/de/anomic/data/wiki/tokens/DefinitionListToken.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdRobotsTxtConfig.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/plasma/dbImport/AbstractImporter.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/portForwarding/sch/serverPortForwardingSch.java, source/de/anomic/server/portForwarding/serverPortForwarding.java, source/de/anomic/server/portForwarding/upnp/serverPortForwardingUpnp.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverPlainSwitch.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFile.java, source/de/anomic/yacy/seedUpload/yacySeedUploadFtp.java, source/de/anomic/yacy/seedUpload/yacySeedUploadScp.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacySeedUploader.java
Sun Mar 16 23:31:54 CET 2008
by orbiter
changed handling of default values and database paths:
- the default files yacy.init and for the network definition is now moved to the path defaults
- the httpProxy.conf is renamed to yacy.conf
- the DATA/INDEX/PUBLIC is renamed to the actual network nickname, which should be freeworld or sciencenet
more menu entries
- added apfelmaennchens alternative search page to the menu
- added the new thread dump page to the server log menu point as submenu
modifications
- modified the thread dump page: sorting by thread type

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4575 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: addon/yacyInit.sh, build.xml, defaults/yacy.init, defaults/yacy.network.freeworld.unit, defaults/yacy.network.group, defaults/yacy.network.intranet.unit, defaults/yacy.network.readme, htroot/ConfigProperties_p.html, htroot/Config_p.html, htroot/IndexImport_p.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/Settings_p.html, htroot/Threaddump_p.html, htroot/Threaddump_p.java, htroot/WatchCrawler_p.java, htroot/env/templates/header.template, htroot/env/templates/submenuViewLog.template, locales/de.lng, locales/fr.lng, locales/it.lng, locales/sk.lng, source/de/anomic/plasma/dbImport/AbstractImporter.java, source/de/anomic/plasma/dbImport/SitemapImporter.java, source/de/anomic/plasma/dbImport/dbImporter.java, source/de/anomic/plasma/dbImport/plasmaCrawlNURLImporter.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/xml/opensearchdescriptionReader.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java, startYACY.bat, startYACY.sh, startYACY_noconsole.bat, stopYACY.sh, yacy-svn-4.spec, yacy.nsi
Sat Mar 15 01:49:16 CET 2008
by orbiter
fixed deadlock

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4562 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/httpc.java, source/de/anomic/http/httpdByteCountInputStream.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlNURL.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverCore.java, source/de/anomic/yacy/yacyClient.java
Fri Mar 14 01:16:04 CET 2008
by orbiter
- next try to fix the networking problem:
  set the maximum transfer size to less than MTU=1500-52: buffer size <= 1448
- some refactoring of transfer methods (naming)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4558 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/WatchCrawler_p.java, htroot/xml/util/getpageinfo_p.java, htroot/yacy/transfer.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/server/serverSystem.java, source/yacy.java
Wed Mar 12 01:05:18 CET 2008
by orbiter
reverted changes to yacy protocol classes: they caused the sciencenet to loose connections
a comparisment with the main release 0.57 had been made: this showed a stable network
This is an emergency operation to ensure availability of the sciencenet network.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4553 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacySeedDB.java
Tue Mar 11 12:09:38 CET 2008
by orbiter
- modified zone navigation (does still not work correctly)
- added dht switch in network definition
- 0.574

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4550 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/yacy/search.java, htroot/yacy/ui/result.java, htroot/yacy/user/sidebar_navigation.html, htroot/yacy/user/sidebar_navigation.java, htroot/yacy/user/ysearch.java, htroot/yacysearch.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyURL.java, yacy.network.unit
Mon Mar 10 00:48:24 CET 2008
by orbiter
some code enhancements and bugfixes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4542 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, source/de/anomic/http/httpd.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBitfield.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroProfile.java, source/de/anomic/net/ftpc.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/server/serverHandler.java, source/de/anomic/tools/disorderSet.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacySeed.java
Sun Mar 09 18:25:09 CET 2008
by apfelmaennchen
fixed the box headings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4541 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/css/base.css, htroot/yacy/ui/css/jquery.tablesorter.pager.css, htroot/yacy/ui/img/box200.png, htroot/yacy/ui/img/box250.png, htroot/yacy/ui/img/box600.png, htroot/yacy/ui/index.html, htroot/yacy/ui/result.class, htroot/yacy/ui/ritem.class, htroot/yacy/ui/welcome.html, htroot/yacy/ui/ymarks.class, htroot/yacy/ui/ymarks.html
Sun Mar 09 13:56:11 CET 2008
by apfelmaennchen
additional images

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4539 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/img-2/accept.png, htroot/yacy/ui/img-2/accept_blue.png, htroot/yacy/ui/img-2/accept_green.png, htroot/yacy/ui/img-2/arrow_2_left_round.png, htroot/yacy/ui/img-2/arrow_2_right_round.png, htroot/yacy/ui/img-2/arrow_left_blue_round.png, htroot/yacy/ui/img-2/arrow_right_blue_round.png, htroot/yacy/ui/img-2/article.png, htroot/yacy/ui/img-2/article_text.png, htroot/yacy/ui/img-2/bar_graph.png, htroot/yacy/ui/img-2/burst.png, htroot/yacy/ui/img-2/calendar.png, htroot/yacy/ui/img-2/calendar2.png, htroot/yacy/ui/img-2/cancel.png, htroot/yacy/ui/img-2/cancel_round.png, htroot/yacy/ui/img-2/clock.png, htroot/yacy/ui/img-2/code.png, htroot/yacy/ui/img-2/comment_blue.png, htroot/yacy/ui/img-2/dollar.png, htroot/yacy/ui/img-2/double_comment.png, htroot/yacy/ui/img-2/download.png, htroot/yacy/ui/img-2/envelope.png, htroot/yacy/ui/img-2/flag_black.png, htroot/yacy/ui/img-2/flag_blue.png, htroot/yacy/ui/img-2/flag_green.png, htroot/yacy/ui/img-2/flag_orange.png, htroot/yacy/ui/img-2/flag_red.png, htroot/yacy/ui/img-2/flag_yellow.png, htroot/yacy/ui/img-2/folder.png, htroot/yacy/ui/img-2/info.png, htroot/yacy/ui/img-2/link.png, htroot/yacy/ui/img-2/lock.png, htroot/yacy/ui/img-2/magnify.png, htroot/yacy/ui/img-2/minus.png, htroot/yacy/ui/img-2/minus_round.png, htroot/yacy/ui/img-2/money.png, htroot/yacy/ui/img-2/new.png, htroot/yacy/ui/img-2/pencil.png, htroot/yacy/ui/img-2/pictures.png, htroot/yacy/ui/img-2/plus.png, htroot/yacy/ui/img-2/printer.png, htroot/yacy/ui/img-2/question_blue.png, htroot/yacy/ui/img-2/refresh.png, htroot/yacy/ui/img-2/rss.png, htroot/yacy/ui/img-2/rss_round.png, htroot/yacy/ui/img-2/security_green.png, htroot/yacy/ui/img-2/security_orange.png, htroot/yacy/ui/img-2/security_red.png, htroot/yacy/ui/img-2/star.png, htroot/yacy/ui/img-2/star_blue.png, htroot/yacy/ui/img-2/stop.png, htroot/yacy/ui/img-2/stop_round.png, htroot/yacy/ui/img-2/upload.png, htroot/yacy/ui/img-2/world.png, htroot/yacy/ui/img-2/world2.png
Sun Mar 09 13:55:26 CET 2008
by apfelmaennchen
additional images

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4538 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/img-1/Add Appointment.png, htroot/yacy/ui/img-1/Add Green Button.png, htroot/yacy/ui/img-1/Add To Favorite.png, htroot/yacy/ui/img-1/Add.png, htroot/yacy/ui/img-1/Appointment Cool.png, htroot/yacy/ui/img-1/Appointment Urgent.png, htroot/yacy/ui/img-1/Audio CD.png, htroot/yacy/ui/img-1/Audio Document.png, htroot/yacy/ui/img-1/Backup Green Button.png, htroot/yacy/ui/img-1/Bandwidth.png, htroot/yacy/ui/img-1/Blue Ball.png, htroot/yacy/ui/img-1/CD.png, htroot/yacy/ui/img-1/Cancel Red Button.png, htroot/yacy/ui/img-1/Clear Green Button.png, htroot/yacy/ui/img-1/Coherence.png, htroot/yacy/ui/img-1/Desktop.png, htroot/yacy/ui/img-1/Discussion.png, htroot/yacy/ui/img-1/Document.png, htroot/yacy/ui/img-1/Donate.png, htroot/yacy/ui/img-1/Eject Blue Button.png, htroot/yacy/ui/img-1/Export To Audio Document.png, htroot/yacy/ui/img-1/Export To Document.png, htroot/yacy/ui/img-1/Export To Movie Document.png, htroot/yacy/ui/img-1/Export To Picture Document.png, htroot/yacy/ui/img-1/Favorite.png, htroot/yacy/ui/img-1/Forward Mail.png, htroot/yacy/ui/img-1/Fullscreen.png, htroot/yacy/ui/img-1/Gear Alt.png, htroot/yacy/ui/img-1/Gear.png, htroot/yacy/ui/img-1/Get Document.png, htroot/yacy/ui/img-1/Get Info Blue Button.png, htroot/yacy/ui/img-1/Get Info Purple Button.png, htroot/yacy/ui/img-1/Get Mail.png, htroot/yacy/ui/img-1/Green Ball.png, htroot/yacy/ui/img-1/Grey Ball.png, htroot/yacy/ui/img-1/Help Blue Button.png, htroot/yacy/ui/img-1/Help Purple Button.png, htroot/yacy/ui/img-1/History.png, htroot/yacy/ui/img-1/Import Audio Document.png, htroot/yacy/ui/img-1/Import Document.png, htroot/yacy/ui/img-1/Import Movie Document.png, htroot/yacy/ui/img-1/Import Picture Document.png, htroot/yacy/ui/img-1/Internet History.png, htroot/yacy/ui/img-1/Mail.png, htroot/yacy/ui/img-1/Menu Item.png, htroot/yacy/ui/img-1/Menu.png, htroot/yacy/ui/img-1/Minus Green Button.png, htroot/yacy/ui/img-1/Minus Red Button.png, htroot/yacy/ui/img-1/Movie CD.png, htroot/yacy/ui/img-1/Movie Document.png, htroot/yacy/ui/img-1/Mr. Bomb.png, htroot/yacy/ui/img-1/Network.png, htroot/yacy/ui/img-1/New Document.png, htroot/yacy/ui/img-1/New Mail.png, htroot/yacy/ui/img-1/Orange Ball.png, htroot/yacy/ui/img-1/Pause All.png, htroot/yacy/ui/img-1/Pause Blue Button.png, htroot/yacy/ui/img-1/Pause Green Button.png, htroot/yacy/ui/img-1/Pause.png, htroot/yacy/ui/img-1/Picture CD.png, htroot/yacy/ui/img-1/Picture Document.png, htroot/yacy/ui/img-1/Play All.png, htroot/yacy/ui/img-1/Play Blue Button.png, htroot/yacy/ui/img-1/Play Green Button.png, htroot/yacy/ui/img-1/Play.png, htroot/yacy/ui/img-1/Plugin Green Button.png, htroot/yacy/ui/img-1/Purple Ball.png, htroot/yacy/ui/img-1/Record Button.png, htroot/yacy/ui/img-1/Record Red Button.png, htroot/yacy/ui/img-1/Red Ball.png, htroot/yacy/ui/img-1/Remove Appointment.png, htroot/yacy/ui/img-1/Remove Document.png, htroot/yacy/ui/img-1/Remove.png, htroot/yacy/ui/img-1/Rename Document.png, htroot/yacy/ui/img-1/Run.png, htroot/yacy/ui/img-1/Send Document.png, htroot/yacy/ui/img-1/Send Mail.png, htroot/yacy/ui/img-1/Smiley Blue.png, htroot/yacy/ui/img-1/Smiley Sad Blue.png, htroot/yacy/ui/img-1/Smiley Sad.png, htroot/yacy/ui/img-1/Smiley Star Pink.png, htroot/yacy/ui/img-1/Smiley Star Sad.png, htroot/yacy/ui/img-1/Smiley Star.png, htroot/yacy/ui/img-1/Smiley.png, htroot/yacy/ui/img-1/Spotlight Blue Button.png, htroot/yacy/ui/img-1/Star.png, htroot/yacy/ui/img-1/Stop All.png, htroot/yacy/ui/img-1/Stop Green Button.png, htroot/yacy/ui/img-1/Stop Red Button.png, htroot/yacy/ui/img-1/Stop.png, htroot/yacy/ui/img-1/Terminal.png, htroot/yacy/ui/img-1/Transfer Document.png, htroot/yacy/ui/img-1/Transfer.png, htroot/yacy/ui/img-1/Trash Empty.png, htroot/yacy/ui/img-1/Trash Full.png, htroot/yacy/ui/img-1/Universal Binary.png, htroot/yacy/ui/img-1/Unread Mail Alt.png, htroot/yacy/ui/img-1/Unread Mail.png, htroot/yacy/ui/img-1/User.png, htroot/yacy/ui/img-1/Users.png, htroot/yacy/ui/img-1/Web Browser.png, htroot/yacy/ui/img-1/Write Document.png, htroot/yacy/ui/img-1/Yellow Ball.png, htroot/yacy/ui/img-1/home.png, htroot/yacy/ui/img-1/iChat Alt.png, htroot/yacy/ui/img-1/iChat.png
Sun Mar 09 13:54:13 CET 2008
by apfelmaennchen
for testing puposes - new user interface based on jQuery and Ajax

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4536 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/js/jquery-1.2.3.min.js, htroot/yacy/ui/js/jquery.dimensions.min.js, htroot/yacy/ui/js/jquery.field.min.js, htroot/yacy/ui/js/jquery.form.js, htroot/yacy/ui/js/jquery.metadata.min.js, htroot/yacy/ui/js/jquery.tablesorter.min.js, htroot/yacy/ui/js/jquery.tablesorter.pager.js, htroot/yacy/ui/js/ui.tabs.ext.min.js, htroot/yacy/ui/js/ui.tabs.min.js
Sun Mar 09 13:53:50 CET 2008
by apfelmaennchen
for testing puposes - new user interface based on jQuery and Ajax

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4535 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/ui/img/asc.gif, htroot/yacy/ui/img/bg.gif, htroot/yacy/ui/img/desc.gif, htroot/yacy/ui/img/first.png, htroot/yacy/ui/img/kaskelix.png, htroot/yacy/ui/img/last.png, htroot/yacy/ui/img/loading.gif, htroot/yacy/ui/img/next.png, htroot/yacy/ui/img/prev.png, htroot/yacy/ui/img/tab.png, htroot/yacy/ui/img/yacy-logo.png
Wed Mar 05 22:46:55 CET 2008
by orbiter
- some refactoring in search process
- separated sidebars in new search interface and placed them in their own files
  which can be put in into the search page like plug-ins

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4529 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, htroot/yacy/user/sidebar_history.html, htroot/yacy/user/sidebar_history.java, htroot/yacy/user/sidebar_navigation.html, htroot/yacy/user/sidebar_navigation.java, htroot/yacy/user/ysearch.html, htroot/yacy/user/ysearchitem.html, htroot/yacy/user/ysearchitem.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/yacy/yacyURL.java
Tue Mar 04 23:45:45 CET 2008
by orbiter
refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4524 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/osm.java, htroot/xml/bookmarks/xbel/xbel.class, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/http/httpc.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBufferedEcoFS.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFlexWidthArray.java, source/de/anomic/kelondro/kelondroMHashMap.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroStack.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaCrawlProfile.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/server/serverCore.java, source/de/anomic/xml/crawlHandler.java, source/de/anomic/xml/rssReader.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/ymage/ymageBMPParser.java, source/screenshot.java
Sun Mar 02 16:42:50 CET 2008
by danielr
consistent use of de.anomic.server.serverMemory to get information about memory statistics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4522 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceMemory_p.java, htroot/Status.java, htroot/xml/status_p.java, source/dbtest.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/server/serverMemory.java, source/yacy.java
Thu Feb 28 00:12:43 CET 2008
by orbiter
- fixed a bug in remote search that prevented that any results had been generated (!)
- added a great number of printStackTrace and new exceptions that shall be used to find the cause
  for a bug in yacy client-server communication which causes the interruption of data transfer
  which then causes the parser bug for the seed strings.
- tried to fix the communication bug on server-side (copy functions)
Be aware that the log may be full of errors and bugs - there should not be more bugs but there is more to see


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4519 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/httpc.java, source/de/anomic/http/httpdByteCountInputStream.java, source/de/anomic/kelondro/kelondroSortStore.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/gzip.java, source/de/anomic/yacy/yacyClient.java
Wed Feb 27 16:16:47 CET 2008
by orbiter
more multithreading support:
- replaced some synchronized classes by classes from util.concurrent
- used a util.concurrent.SynchronousQueue to implement a persistent sorting thread in
  the very basic kelondroRowCollection which supports sorting with a second thread
  in case that a double-core processing CPU is used

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4517 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Status.java, htroot/xml/status_p.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroSortStack.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverDomains.java, source/de/anomic/server/serverProcessor.java, source/de/anomic/server/serverProfiling.java, source/yacy.java
Mon Feb 25 22:26:49 CET 2008
by orbiter
added a search history to the new search page
the history distinguishes between different users and identifies them by their ip
a history is only shown to the user who submitted the search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4510 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlProfileEditor_p.html, htroot/yacy/user/ysearch.html, htroot/yacy/user/ysearchitem.html, htroot/yacy/user/ysearchitem.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/yacy/yacyClient.java
Mon Feb 25 15:08:15 CET 2008
by orbiter
- enhanced recognition, parsing, management and double-occurrence-handling of image tags
- enhanced text parser (condenser): found and eliminated bad code parts; increase of speed
- added handling of image preview using the image cache from HTCACHE
- some other minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4507 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CacheAdmin_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlNURL.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSnippetCache.java
Fri Feb 22 00:40:38 CET 2008
by orbiter
- fixed some data types in new search stack
- added image domain presentation to image preview
- added new search page to menu
- added automatic re-search when an old search profile is requested and a crawl is ongoing,
  to fetch newly crawled entries

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4501 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/env/templates/header.template, htroot/yacy/user/ysearch.java, htroot/yacy/user/ysearchitem.html, htroot/yacy/user/ysearchitem.java, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/kelondro/kelondroSortStack.java, source/de/anomic/kelondro/kelondroSortStore.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java
Thu Feb 21 15:53:51 CET 2008
by orbiter
- added image sorting by image size. This is the default now.
  This is performed using a 3-stage sorting process:
  - sort by relevance, then do snippet-fetch
  - sort snippets by relevance then do image link extraction
  - sort image links by image size; unknown sizes are handled like small sizes
- only the exact amount of images as requested are shown

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4499 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/user/ysearchitem.html, htroot/yacy/user/ysearchitem.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/htmlFilter/htmlFilterImageEntry.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSnippetCache.java
Thu Feb 21 11:06:57 CET 2008
by orbiter
- fixed some bugs in ranking computation
- introduced generalized method to organize ranked results (2 new classes)
- added a post-ranking after snippet-fetch (before: only listed) using the new ranking data structures
- fixed some missing data fields in RWI ranking attributes and correct hand-over between data structures

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4498 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/search.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/kelondro/kelondroSortStack.java, source/de/anomic/kelondro/kelondroSortStore.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, yacy.network.group
Tue Feb 19 15:13:35 CET 2008
by orbiter
- fixed highslide usage
- some enhancement to index management, better types

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4497 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/js/highslide/graphics/controlbar2.gif, htroot/js/highslide/graphics/controlbar3.gif, htroot/js/highslide/graphics/controlbar4-hover.gif, htroot/js/highslide/graphics/controlbar4.gif, htroot/js/highslide/graphics/fullexpand.gif, htroot/js/highslide/graphics/geckodimmer.png, htroot/js/highslide/graphics/loader.gif, htroot/js/highslide/graphics/loader.white.gif, htroot/js/highslide/graphics/outlines/Outlines.psd, htroot/js/highslide/graphics/outlines/beveled.png, htroot/js/highslide/graphics/outlines/drop-shadow.png, htroot/js/highslide/graphics/outlines/glossy-dark.png, htroot/js/highslide/graphics/outlines/outer-glow.png, htroot/js/highslide/graphics/outlines/rounded-black.png, htroot/js/highslide/graphics/outlines/rounded-white.png, htroot/js/highslide/graphics/resize.gif, htroot/js/highslide/graphics/zoomin.cur, htroot/js/highslide/graphics/zoomout.cur, htroot/js/highslide/highslide.js, htroot/yacy/transferRWI.java, htroot/yacy/user/ysearch.html, htroot/yacysearch.html, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java
Tue Feb 19 10:14:07 CET 2008
by orbiter
- changed order of database initialisation (index first)
- removed mainly unused init-time for databases (was only used for tree tables, which are not used any more)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4496 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroDynTree.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/dbImport/plasmaCrawlNURLImporter.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaCrawlProfile.java, source/de/anomic/plasma/plasmaCrawlRobotsTxt.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordConnotation.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacySeedDB.java, source/migration.java, source/yacy.java, yacy.init
Tue Feb 19 00:49:03 CET 2008
by orbiter
- some small changes to highslide integration to get it working... (does not work yet)
- performance enhancement for url list parser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4495 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, htroot/env/highslide.css, htroot/js/highslide.js, htroot/yacy/user/default.css, htroot/yacy/user/ysearch.html, htroot/yacy/user/ysearchitem.html, htroot/yacysearch.html, htroot/yacysearchitem.html, source/de/anomic/plasma/plasmaParser.java
Mon Feb 18 17:38:06 CET 2008
by orbiter
- added new way to watch images from the image search:
  they appear as separate, floating window above the search results,
  not in a new window
- added highslide javascript library for feature mentioned above
- removed dir servlet. This thing was not used as it was supposed to be (as an example applet)
  and was a major problem for intranet-indexing when files are hosted on the same peer.
- added yacy-httpd-internal directory listing. Because YaCy is a search engine,
  directory listings are similar to search result listings. Intranet indexing from the same peer
  will get nice index pages for document collections.
- removed unused test applet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4494 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, htroot/env/highslide.css, htroot/js/highslide.js, htroot/yacy/user/ysearch.html, htroot/yacy/user/ysearchitem.html, htroot/yacysearch.html, htroot/yacysearchitem.html, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/httpdFileHandler.java, source/yacy.java
Sat Feb 16 11:36:10 CET 2008
by orbiter
added images folder


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4484 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/user/images/arrow1.gif, htroot/yacy/user/images/arrow2.gif, htroot/yacy/user/images/background.gif, htroot/yacy/user/images/house.gif, htroot/yacy/user/images/img01.gif, htroot/yacy/user/images/img03.gif, htroot/yacy/user/images/img04.gif, htroot/yacy/user/images/img05.gif, htroot/yacy/user/images/img08.gif, htroot/yacy/user/images/img09.gif, htroot/yacy/user/images/letter.gif, htroot/yacy/user/images/spacer.gif, htroot/yacy/user/images/tree.gif
Sat Feb 16 01:38:47 CET 2008
by orbiter
first test of alternative search interface (only a stub but working!)
try http://localhost:8080/yacy/user/ysearch.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4482 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: .classpath, htroot/yacy/user/default.css, htroot/yacy/user/license.txt, htroot/yacy/user/ysearch.html, htroot/yacy/user/ysearch.java, htroot/yacy/user/ysearchitem.html, htroot/yacy/user/ysearchitem.java, htroot/yacysearch/images/arrow1.gif, htroot/yacysearch/images/arrow2.gif, htroot/yacysearch/images/background.gif, htroot/yacysearch/images/house.gif, htroot/yacysearch/images/letter.gif, htroot/yacysearch/images/tree.gif
Fri Feb 15 14:12:25 CET 2008
by orbiter
modified dir servlet to cooperate with intranet indexing from the own HTDOCS repository:
- removed md5 file generation (spoils the won repository)
- removed comments in file share (was never used)
- moved dir list comparator to other place (maybe solves problem, lets see)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4481 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/htdocsdefault/dir.html, htroot/htdocsdefault/dir.java, htroot/htdocsdefault/dir.rss, htroot/htdocsdefault/dir.xml, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/server/serverCodings.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/ymage/ymageOSM.java
Fri Feb 08 15:06:34 CET 2008
by borg-0300
seedDB helpers update/cleanup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4461 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacySeed.java
Thu Feb 07 23:16:36 CET 2008
by orbiter
- faster search: using different data structures that avoid multiplr calculations
- no more table copy for error-eco table
- optional table copy for lurl-entries
- more abstractions (less single constant strings)
- better logging (using host names instead of ips)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4459 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlogComments.java, htroot/CrawlProfileEditor_p.java, htroot/Settings_p.java, htroot/TestApplet.java, htroot/User.java, htroot/ViewImage.java, htroot/Wiki.java, htroot/www/welcome.java, htroot/yacy/hello.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacysearch.java, source/de/anomic/data/userDB.java, source/de/anomic/http/httpSSI.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java
Tue Feb 05 00:32:19 CET 2008
by orbiter
added stub for new search page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4446 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/yacysearch/default.css, htroot/yacysearch/images/img01.gif, htroot/yacysearch/images/img03.gif, htroot/yacysearch/images/img04.gif, htroot/yacysearch/images/img05.gif, htroot/yacysearch/images/img08.gif, htroot/yacysearch/images/img09.gif, htroot/yacysearch/images/spacer.gif, htroot/yacysearch/index.html, htroot/yacysearch/license.txt
Sun Feb 03 21:11:50 CET 2008
by orbiter
- fix for online caution delay settings, see
  http://forum.yacy-websuche.de/viewtopic.php?f=6&t=738&p=4723#p4723
- removed remote search limitation for non-dht-peers according to discussion in
  http://forum.yacy-websuche.de/viewtopic.php?f=15&t=793&hilit=&p=5277#p5277

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4438 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, htroot/index.java, htroot/yacysearch.java, source/de/anomic/server/serverSystem.java, yacy.init
Sun Feb 03 00:53:39 CET 2008
by orbiter
- no more search time-out!
- fixed a bug with last commit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4430 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/index.html, htroot/index.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRI.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java
Fri Feb 01 00:40:47 CET 2008
by orbiter
- fixed bug with too much RAM in crawler queue
- fixed dir bug
- better calculation of TF for join
- better waiting-on-result logic

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4424 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, source/de/anomic/http/httpdSwitchboard.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/plasma/plasmaCrawlNURL.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/logging/serverLog.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverPlainSwitch.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyVersion.java, source/migration.java, source/yacy.java
Wed Jan 30 22:58:30 CET 2008
by orbiter
- more dublin core naming of page metadata
- better presentation of result counters in search results

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4420 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/ViewFile.java, htroot/js/yacysearch.js, htroot/yacy/search.java, htroot/yacy/urls.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyClient.java
Wed Jan 30 01:15:43 CET 2008
by orbiter
fixed bug in remote search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4419 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaWordIndex.java
Tue Jan 29 17:41:09 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4415 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/BlacklistCleaner_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigLanguage_p.java, htroot/ConfigProperties_p.java, htroot/ConfigSkins_p.java, htroot/Config_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieTest.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlStartSimple_p.java, htroot/IndexControlRWIs_p.html, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexImport_p.java, htroot/IndexTransfer_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/PeerLoadPicture.java, htroot/PerformanceSearch_p.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Supporter.java, htroot/Surftips.java, htroot/TestApplet.java, htroot/User_p.java, htroot/ViewImage.java, htroot/ViewProfile.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/Wiki.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/xml/blacklists_p.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/get.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/xbel/xbel.java, htroot/xml/config_p.java, htroot/xml/queues_p.java, htroot/xml/webstructure.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java
Tue Jan 29 11:12:48 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4414 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Blacklist_p.java, htroot/News.java, htroot/SettingsAck_p.java, htroot/Statistics.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/URLLicense.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/diff.java, source/de/anomic/data/listManager.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/robotsParser.java, source/de/anomic/data/translator.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/data/wikiCode.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpc.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroMapObjects.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/kelondro/kelondroObjectsMapEntry.java, source/de/anomic/kelondro/kelondroRA.java, source/de/anomic/plasma/cache/IResourceInfo.java, source/de/anomic/plasma/cache/ftp/ResourceInfo.java, source/de/anomic/plasma/cache/http/ResourceInfo.java, source/de/anomic/plasma/dbImport/AbstractImporter.java, source/de/anomic/plasma/dbImport/SitemapImporter.java, source/de/anomic/plasma/dbImport/dbImportManager.java, source/de/anomic/plasma/dbImport/dbImporter.java, source/de/anomic/plasma/dbImport/plasmaCrawlNURLImporter.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/parser/bzip/bzipParser.java, source/de/anomic/plasma/parser/doc/docParser.java, source/de/anomic/plasma/parser/gzip/gzipParser.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/mimeType/odtDetector.java, source/de/anomic/plasma/parser/mimeType/rssDetector.java, source/de/anomic/plasma/plasmaCrawlProfile.java, source/de/anomic/plasma/plasmaCrawlRobotsTxt.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/server/serverCodings.java, source/de/anomic/server/serverObjects.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java
Mon Jan 28 21:08:32 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4412 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CacheAdmin_p.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/plasma/cache/ResourceInfoFactory.java, source/de/anomic/plasma/cache/ftp/ResourceInfo.java, source/de/anomic/plasma/cache/http/ResourceInfo.java, source/de/anomic/plasma/crawler/plasmaProtocolLoader.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java
Mon Jan 28 19:21:08 CET 2008
by orbiter
- added ViewProfile as rdf in foaf syntax
- added link to rdf and vCard version on html page
- can be seen on http://localhost:8080/ViewProfile.html?hash=localhash
- more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4411 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/ViewProfile.html, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/data/wiki/tokens/LinkToken.java, source/de/anomic/data/wiki/tokens/ListToken.java, source/de/anomic/data/wiki/tokens/SimpleToken.java, source/de/anomic/data/wiki/tokens/TableToken.java, source/de/anomic/htmlFilter/htmlFilterAbstractScraper.java, source/de/anomic/htmlFilter/htmlFilterAbstractTransformer.java, source/de/anomic/htmlFilter/htmlFilterContentTransformer.java, source/de/anomic/htmlFilter/htmlFilterImageEntry.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpRemoteProxyConfig.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdByteCountInputStream.java, source/de/anomic/http/httpdByteCountOutputStream.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/http/httpdSwitchboard.java, source/de/anomic/icap/icapHeader.java, source/de/anomic/net/ftpc.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaRankingCRProcess.java
Fri Jan 25 12:44:27 CET 2008
by orbiter
added termination control for RotateIterator

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4399 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlURLs_p.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/kelondro/kelondroObjects.java, source/de/anomic/kelondro/kelondroRotateIterator.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyDHTAction.java
Thu Jan 24 23:49:00 CET 2008
by orbiter
- refactoring of indexControlRWIs: moved statics to own class; better Dublin Core naming
- fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=759&hilit=&p=4866#p4866
- some bugfixes in EcoTable according remove method
- switched more tables to Eco: crawl Profiles, htcache, seeddb, newsdb

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4397 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, source/de/anomic/http/httpc.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroRotateIterator.java, source/de/anomic/plasma/dbImport/plasmaCrawlNURLImporter.java, source/de/anomic/plasma/plasmaCrawlProfile.java, source/de/anomic/plasma/plasmaCrawlRobotsTxt.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSearchAPI.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsPool.java, yacy.init
Thu Jan 24 00:08:32 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4382 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/plasma/urlPattern/abstractURLPattern.java, source/de/anomic/plasma/urlPattern/defaultURLPattern.java, source/de/anomic/plasma/urlPattern/plasmaURLPattern.java, source/de/anomic/server/logging/ConsoleOutErrHandler.java, source/de/anomic/server/logging/GuiHandler.java, source/de/anomic/server/logging/LogalizerHandler.java, source/de/anomic/server/logging/logParsers/LogParserPLASMA.java, source/de/anomic/yacy/yacyNewsPool.java, source/de/anomic/yacy/yacyNewsQueue.java
Wed Jan 23 22:09:56 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4379 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/htdocsdefault/dir.java, source/de/anomic/tools/PKCS12Tool.java, source/de/anomic/tools/cryptbig.java, source/de/anomic/tools/disorderHeap.java, source/de/anomic/tools/disorderSet.java, source/de/anomic/tools/enumerateFiles.java, source/de/anomic/tools/loaderThreads.java, source/de/anomic/xml/crawlHandler.java, source/de/anomic/xml/rssReader.java
Tue Jan 22 12:51:43 CET 2008
by orbiter
- added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser
- refactoring of plasmaParserDocument to use Dublin Core - compatible property names
- redesign of url handling in parser and condenser (less String-to-yacyURL conversion)
- more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4352 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/ViewFile.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/plasma/parser/Parser.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/rtf/rtfParser.java, source/de/anomic/plasma/parser/sevenzip/SZParserExtractCallback.java, source/de/anomic/plasma/parser/sevenzip/sevenzipParser.java, source/de/anomic/plasma/parser/swf/swfParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/xls/xlsParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaCrawlProfile.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyURL.java, source/de/anomic/yacy/yacyVersion.java
Sun Jan 20 22:42:35 CET 2008
by orbiter
- extended dbtest for comparisment tests
- added initial space option for eco tables
- used initial space value in initialization of collectionIndex, this should avoid OOM failures" /Volumes/Magneto/dev/workspace/trunk/source/dbtest.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroCollectionIndex.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroDyn.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroEcoTable.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroRow.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/kelondro/kelondroSplitTable.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlBalancer.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlStacker.java /Volumes/Magneto/dev/workspace/trunk/source/de/anomic/plasma/plasmaCrawlZURL.java
- added index consistency check (checks for double-occurrences of primary keys in file)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4349 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java
Sun Jan 20 02:22:46 CET 2008
by orbiter
to fix inconsistencies in collection index, a double reference reporting mechanism has been implemented


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4347 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroTree.java
Sat Jan 19 13:23:56 CET 2008
by orbiter
fixed some problems with eco tables

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4346 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java
Sat Jan 19 01:40:19 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4343 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: AUTHORS, htroot/CookieMonitorOutgoing_p.java, htroot/Settings_p.java, source/de/anomic/kelondro/kelondroCloneableMapIterator.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/kelondro/kelondroSplittedTree.java, source/de/anomic/kelondro/kelondroXMLTables.java, source/de/anomic/plasma/cache/IResourceInfo.java, source/de/anomic/plasma/parser/ParserInfo.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaSearchImages.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/ymage/ymageGraph.java, source/de/anomic/ymage/ymageMatrix.java, source/de/anomic/ymage/ymageToolCircle.java
Fri Jan 18 18:14:02 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4341 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaParserDocument.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java
Thu Jan 17 22:48:08 CET 2008
by orbiter
activated new kelondroEcoTable file structure.
This data structure replaces almost all files in the PLASMA directory
also the collection.index and the LURL-db will be created as Eco-DB, if it does not exist before
existing Flex-databases will be used as they are (the is no data lost)
If you want to force the creation of a Eco-collection.index, simply delete the old index.
The Eco file system will only be used if there is enough memory.
The collection.index RAM limit is 200MB, if you have less, a flex-Table is createt.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4340 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, htroot/PerformanceMemory_p.xml, source/dbtest.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroSplitTable.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java
Thu Jan 17 13:12:52 CET 2008
by orbiter
- removed write buffer from kelondroCache (was never used because buggy; will now be replaced by new EcoBuffer)
- added new data structure 'eco' for an index file that should use only 50% of write-IO compared to kelondroFlex
The new eco index is not used yet, but already successfully tested with the collectionIndex
The main purpose is to replace the kelondroFlex at every point when enough RAM is available.
Othervise, the kelondroFlex stays as option in case of low memory (which then can even use a file-index)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4337 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBufferedEcoFS.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroEcoFS.java, source/de/anomic/kelondro/kelondroEcoTable.java, source/de/anomic/kelondro/kelondroFlexSplitTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyNewsDB.java
Sat Jan 12 01:24:24 CET 2008
by borg-0300
more information (BASE64)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4324 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/userDB.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBitfield.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/tools/crypt.java, source/de/anomic/yacy/yacySeed.java, yacy.logging
Fri Jan 11 01:12:01 CET 2008
by orbiter
- added large number of generics
- redesign of ordering structures in kelondro (old did not work with strict generics)
- 50% IO reduction during read access on kelondroFlex (ommiting of read on index table)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4320 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/hello.java, htroot/yacy/profile.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/dbtest.java, source/de/anomic/index/indexCachedRI.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/kelondro/kelondroAbstractOrder.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBinSearch.java, source/de/anomic/kelondro/kelondroByteArray.java, source/de/anomic/kelondro/kelondroByteOrder.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCloneableMapIterator.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroFlexSplitTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFlexWidthArray.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMapObjects.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroNaturalOrder.java, source/de/anomic/kelondro/kelondroOrder.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/kelondro/kelondroRotateIterator.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplittedTree.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/kelondro/kelondroXMLTables.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacySeedDB.java
Thu Jan 10 00:05:52 CET 2008
by orbiter
- removed pro-version (is the same as standard version, use the standard instead)
- changed yacy logo
- removed crawlOrder protocol (unused)
- removed file index in kelondroFlex (will not work, it takes too long to maintain)
- fixed remoted crawl for clusters (now denies remote crawls from peers outside cluster)
- 0.562

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4317 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, build.xml, htroot/env/grafics/YaCyLogo_120ppi.png, htroot/env/grafics/YaCyLogo_60ppi.png, htroot/env/grafics/yacy.gif, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/yacy/yacyDHTAction.java
Wed Jan 09 10:58:56 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4311 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexCachedRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRI.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroFlexSplitTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroMHashMap.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMapObjects.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/server/serverClassLoader.java, source/de/anomic/server/serverInstantThread.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/server/serverSystem.java
Tue Jan 08 21:12:31 CET 2008
by orbiter
- added NEAR operator (must be written in UPPERCASE in search query)
- more generics
- removed unused commons classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4310 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/Ranking_p.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java
Mon Jan 07 23:36:48 CET 2008
by orbiter
- some refactoring and redesign of kelondroBytesIntMap (created new class kelondroRAMIndex)
- more generics
- preparation to extend the balancer for flexible forced delay times
- set different random-access type, should now omit update of metadata in file and could be a bit faster (lets see)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4309 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroFlexWidthArray.java, source/de/anomic/kelondro/kelondroObjectSpace.java, source/de/anomic/kelondro/kelondroRAMIndex.java, source/de/anomic/plasma/plasmaCrawlBalancer.java
Sun Jan 06 20:23:38 CET 2008
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4305 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Blog.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/Connections_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlStartSimple_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexTransfer_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, htroot/PerformanceQueues_p.xml, htroot/Ranking_p.java, htroot/Statistics.java, htroot/Status.java, htroot/Status_p.inc, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/robots.java, htroot/www/welcome.java, htroot/yacy/transfer.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpc.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/index/indexCachedRI.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDynTree.java, source/de/anomic/kelondro/kelondroEcoRecords.java, source/de/anomic/kelondro/kelondroFileRA.java, source/de/anomic/kelondro/kelondroFixedWidthArray.java, source/de/anomic/kelondro/kelondroFlexSplitTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/plasma/plasmaCrawlProfile.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/logging/LogalizerHandler.java, source/de/anomic/server/logging/logParsers/LogParser.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverCodings.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverObjects.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/server/servletProperties.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyURL.java, source/yacy.java, yacy.init
Fri Dec 28 19:47:45 CET 2007
by orbiter
more generics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4295 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Supporter.java, htroot/Surftips.java, source/de/anomic/htmlFilter/htmlFilterTransformer.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpd.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMSetTools.java, source/de/anomic/kelondro/kelondroMapObjects.java, source/de/anomic/kelondro/kelondroObjects.java, source/de/anomic/kelondro/kelondroObjectsMapEntry.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroStack.java, source/de/anomic/kelondro/kelondroXMLTables.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/server/serverDomains.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Fri Dec 28 04:39:36 CET 2007
by orbiter
more generics, bugfixes for wrong cast

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4294 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/index/indexCachedRI.java, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexRI.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaWordIndex.java
Thu Dec 27 23:37:02 CET 2007
by low012
*) tried to get rid of warnings when compiling parsers (http://forum.yacy-websuche.de/viewtopic.php?t=660)

lots of warnings are gone, new one in htmlFilterContentScraper


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4293 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/plasma/parser/bzip/bzipParser.java, source/de/anomic/plasma/parser/doc/docParser.java, source/de/anomic/plasma/parser/gzip/gzipParser.java, source/de/anomic/plasma/parser/mimeType/mimeTypeParser.java, source/de/anomic/plasma/parser/odt/odtParser.java, source/de/anomic/plasma/parser/pdf/pdfParser.java, source/de/anomic/plasma/parser/ppt/pptParser.java, source/de/anomic/plasma/parser/ps/psParser.java, source/de/anomic/plasma/parser/rpm/rpmParser.java, source/de/anomic/plasma/parser/rss/rssParser.java, source/de/anomic/plasma/parser/rtf/rtfParser.java, source/de/anomic/plasma/parser/sevenzip/SZParserExtractCallback.java, source/de/anomic/plasma/parser/sevenzip/sevenzipParser.java, source/de/anomic/plasma/parser/swf/swfParser.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/vcf/vcfParser.java, source/de/anomic/plasma/parser/xls/xlsParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserDocument.java
Thu Dec 27 18:56:59 CET 2007
by orbiter
moved to Java 1.5:
- changed build script to use java 1.5 compiler
- first stept to resolve missing generics definition (about 400 from over 4100 'missing'-warnings)
- added key-iterator to kelondro databases (for rapid from-memory enumerations, will be used for domain name collection, not used yet)

please set your development environment to use java 1.5!


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4292 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, source/de/anomic/index/indexCollectionRI.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexContainerOrder.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/kelondro/kelondroAbstractOrder.java, source/de/anomic/kelondro/kelondroAbstractRA.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBufferedIOChunks.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRA.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroCloneableIterator.java, source/de/anomic/kelondro/kelondroCloneableMapIterator.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroFlexSplitTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroHandle.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroMScoreCluster.java, source/de/anomic/kelondro/kelondroMergeIterator.java, source/de/anomic/kelondro/kelondroNaturalOrder.java, source/de/anomic/kelondro/kelondroObjects.java, source/de/anomic/kelondro/kelondroOrder.java, source/de/anomic/kelondro/kelondroRA.java, source/de/anomic/kelondro/kelondroRotateIterator.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplittedTree.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/tools/nxTools.java
Fri Dec 21 01:53:46 CET 2007
by fuchsi
last cleanup bits to serverDate before the release. only safe refactoring (method renaming) changes outside of serverDate.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4289 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/Connections_p.java, htroot/IndexControlURLs_p.java, htroot/IndexImport_p.java, htroot/Network.java, htroot/News.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/rct_p.java, htroot/yacy/query.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, source/de/anomic/data/blogBoard.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/server/serverDate.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsDB.java, source/de/anomic/yacy/yacyNewsQueue.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySeed.java
Thu Dec 20 18:11:35 CET 2007
by hermens
Some enhancements to time management:
- remove unnecessary generation of Calendar and Date objects
- synchronized SimpleDateFormat objects in blog-, message- and wikiBoard
- correct use of TimeZones and SimpleDateFormats



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4288 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/messageBoard.java, source/de/anomic/data/wikiBoard.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpc.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/net/ftpc.java
Thu Dec 20 03:46:41 CET 2007
by orbiter
new release strategy: the standard release is now built the same way as the pro release
a new release type was added: 'embedded' which is the same as the current standard release was
this will not have any effect to the next release 0.56, which will still a pro-release on public download
the transition the the new release strategy must be done now to enable automatic update by the updated in future releases

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4287 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, build.xml, htroot/ConfigUpdate_p.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/yacy/yacyVersion.java, yacy.init
Wed Dec 19 20:39:19 CET 2007
by fuchsi
more cleanup in serverDate
- remove direct accesses to SimpleDateFormat fields in serverDate and use the static parse... methods instead
- remove nowDate() as a Date doesn't store timezone information and a new Date() is always faster
- default formatter methods use a GMT timezone by default now, this is important for interchangability as some date formats we use don't include a timezone offset.
- continued renaming and rearanging (formatter) methods. all should follow the general naming scheme formatWHAT(...)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4285 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacy/transferURL.java, source/de/anomic/data/blogBoard.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/plasma/plasmaWebStructure.java, source/de/anomic/server/serverDate.java, source/yacy.java
Fri Dec 14 20:17:54 CET 2007
by fuchsi
small cosmetic change for static fields in serverCore (special protocol ASCII entities) to improve readability

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4275 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/yacy/hello.java, htroot/yacy/search.java, source/de/anomic/data/listManager.java, source/de/anomic/data/translator.java, source/de/anomic/data/wikiCode.java, source/de/anomic/http/httpChunkedOutputStream.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/http/httpc.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverSystem.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeedDB.java, source/de/anomic/yacy/yacyVersion.java, source/yacy.java
Tue Dec 11 16:07:03 CET 2007
by orbiter
added memory measurement for index recreation to avoid OOM during index RAM space extension

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4267 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroFlexSplitTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/plasma/plasmaRankingCRProcess.java
Fri Dec 07 03:15:12 CET 2007
by orbiter
some memory enhancements when generating and displaying ymage objects

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4253 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Status.html, source/de/anomic/http/httpChunkedInputStream.java, source/de/anomic/http/httpc.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/parser/tar/tarParser.java, source/de/anomic/plasma/parser/zip/zipParser.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/server/serverByteBuffer.java, source/de/anomic/server/serverCore.java, source/de/anomic/ymage/ymageChart.java, source/de/anomic/ymage/ymageMatrix.java
Thu Dec 06 02:06:17 CET 2007
by orbiter
removed CMY color model from ymage classes and re-introduced RGB color model

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4249 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.html, htroot/PeerLoadPicture.java, htroot/imagetest.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/ymage/ymageCaptcha.java, source/de/anomic/ymage/ymageChart.java, source/de/anomic/ymage/ymageDemoApplet.java, source/de/anomic/ymage/ymageGraph.java, source/de/anomic/ymage/ymageMatrix.java
Tue Dec 04 21:19:13 CET 2007
by orbiter
changed method of profiling


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4248 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.html, htroot/PerformanceSearch_p.html, htroot/PerformanceSearch_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaProfiling.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/serverProfiling.java
Mon Dec 03 03:35:28 CET 2007
by orbiter
- generalized ymage initialization options
- auto-adoption of performance memory graph to needed dimension

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4246 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Banner.java, htroot/PerformanceGraph.java, htroot/SearchEventPicture.java, htroot/WebStructurePicture_p.java, htroot/imagetest.java, source/dbtest.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/ymage/ymageCaptcha.java, source/de/anomic/ymage/ymageChart.java, source/de/anomic/ymage/ymageDemoApplet.java, source/de/anomic/ymage/ymageGraph.java, source/de/anomic/ymage/ymageMatrix.java
Thu Nov 29 03:07:37 CET 2007
by orbiter
redesigned remote crawl process:
- instead of pushing urls to other peers, the urls are actively pulled
  by the peer that wants to do a remote crawl
- the remote crawl push process had been removed
- a process that adds urls from remote peers had been added
- the server-side interface for providing 'limit'-urls exists since 0.55 and works with this version
- the list-interface had been removed
- servlets using the list-interface had been removed (this implementation did not properly manage double-check)
- changes in configuration file to support new pull-process
- fixed a bug in crawl balancer (status was not saved/closed properly)
- the yacy/urls-protocol was extended to support different networks/clusters
- many interface-adoptions to new stack counters

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4232 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/PerformanceQueues_p.java, htroot/ScreenSaver.java, htroot/Status.java, htroot/Status_p.inc, htroot/WatchCrawler_p.html, htroot/WatchCrawler_p.java, htroot/js/WatchCrawler.js, htroot/rct_p.html, htroot/rct_p.java, htroot/xml/queues_p.java, htroot/xml/queues_p.xml, htroot/yacy/crawlOrder.java, htroot/yacy/urls.java, htroot/yacy/urls.xml, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/xml/rssReader.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyVersion.java, yacy.init
Thu Nov 22 02:34:29 CET 2007
by orbiter
fixed image search and favicon loading

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4225 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/crawler/plasmaProtocolLoader.java, source/de/anomic/plasma/plasmaCrawlEntry.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaParserConfig.java, source/de/anomic/plasma/plasmaSwitchboard.java, yacy.init
Thu Nov 22 00:14:57 CET 2007
by orbiter
redesign of ranking data structure
- the index administration now uses the same code base for url selection and collection
  as the search interface. The index administration is therefore a good test environment for
  ranking order control
- removed old postsorting-algorithms, will be replaced with new one
- fixed many bugs occurred before during ranking; especially the contraint filtering method
  removed too many links
- fixed media search flags; had been attached to too many urls. The effect should be a better
  pre-sorting before media load within snippet fetch

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4223 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlProfileEditor_p.html, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java
Sat Nov 17 02:53:02 CET 2007
by orbiter
- some enhancements to IndexControlURLs (shows more links, connects referrer to another query)
- some refactoring to search process

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4222 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, htroot/PerformanceSearch_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/index/indexContainer.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/serverProfiling.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySeed.java
Fri Nov 16 15:48:09 CET 2007
by orbiter
- enhanced monitoring of ranking parameters
  for details, please try http://localhost:8080/IndexControlRWIs_p.html
- fixed computation of ranking ordering in some cases

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4220 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/Ranking_p.java, htroot/ViewFile.java, htroot/yacy/crawlOrder.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/http/httpc.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/plasma/dbImport/plasmaDbImporter.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyURL.java, source/yacy.java
Thu Nov 15 04:03:18 CET 2007
by orbiter
preparations for better ranking: better debugging of index properties
to do this, the index administration interface was extended.
It is now possible to select parts of a index.
See properties shown in interface after a word search for details.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4218 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, htroot/env/templates/header.template, htroot/env/templates/submenuIndexControl.template, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaWordIndex.java
Fri Nov 09 01:51:38 CET 2007
by orbiter
enhanced computing speed of kelondro core function: sorting
the enhancement was made by using better organized data structures and
multi-threading during the sort. A sort can be divided into two separate
processes when the first partition of the quicksort algorithm was done.
Generating a separate thread and starting the thread takes only 10 milliseconds,
so using a separate thread makes only sense if the data amount is large.
statistics about the speed-up:
without ehancement: 250 milliseconds for 100000 entries
with data structure enhancement: 170 milliseconds for 100000 entries
with additional second thread (if second processor is present): 130 milliseconds.

For dual-processor systems, this means about 100% speed-up
a test can be made with the following command:
java -classpath classes de.anomic.kelondro.kelondroRowCollection


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4198 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/kelondro/kelondroBase64Order.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/plasma/plasmaWordIndex.java
Wed Nov 07 23:38:09 CET 2007
by orbiter
enhanced local search speed. The ranking process is now 6 times faster that before.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4197 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/CrawlURLFetch_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/yacy/crawlOrder.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/index/indexContainer.java, source/de/anomic/index/indexRAMRI.java, source/de/anomic/index/indexRWIEntry.java, source/de/anomic/index/indexRWIEntryOrder.java, source/de/anomic/index/indexRWIRowEntry.java, source/de/anomic/index/indexRWIVarEntry.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroAttrSeq.java, source/de/anomic/kelondro/kelondroBinSearch.java, source/de/anomic/kelondro/kelondroBitfield.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCachedRecords.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroColumn.java, source/de/anomic/kelondro/kelondroEcoRecords.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroHashtable.java, source/de/anomic/kelondro/kelondroNaturalOrder.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/kelondro/kelondroRowCollection.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/plasma/plasmaRankingCRProcess.java, source/de/anomic/plasma/plasmaRankingRCIEvaluation.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchProcessing.java, source/de/anomic/plasma/plasmaSearchRankingProcess.java, source/de/anomic/plasma/plasmaSearchRankingProfile.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacyURL.java
Sun Nov 04 11:36:25 CET 2007
by fuchsi
Allow absolute paths in configuration settings.
- before absolute paths would be expanded incorrectly, e.g.: fooPath=/a/b/c would become /path/to/yacy/root/a/b/c. Now you can put nearly every dynamically generated data with a configurable path to a location outside of yacys root dir without having to use symlinks (probably good for third party distribution packaging).
- abstractServerSwitch.getConfigPath(setting, default) returns a File instance, either with an absolute path or relative to the applications root path.

- exceptions (hardcoded): 
  DATA/LOG/yacy.logging
  DATA/SETTINGS/httpProxy.conf
  DATA/SETTINGS/user.db
TODO: all of these are the global configuration files and they should probably be put into _one_ command line configurable settings path, so it would be possible to package them in /etc/ for example.

- add missing workPath to yacy.init (it was used in code, but there was no default in the file)
- fix broken skinPath (was skinsPath in yacy.init but skinsPath in the code) + a few other broken config reading caused by typos.
- replaced path setting names and their default values with the related static fields in plasmaSwitchboard where not already done/existing

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4196 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.java, htroot/BlogComments.java, htroot/CacheAdmin_p.java, htroot/CacheResource_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigSkins_p.java, htroot/CrawlURLFetchStack_p.java, htroot/Messages_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/htdocsdefault/dir.java, htroot/www/welcome.java, htroot/yacy/list.java, htroot/yacy/message.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/translator.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/server/logging/serverLog.java, source/de/anomic/server/serverAbstractSwitch.java, source/de/anomic/server/serverSwitch.java, source/de/anomic/yacy/yacyCore.java, source/migration.java, source/yacy.java, yacy.init
Wed Oct 31 12:32:40 CET 2007
by orbiter
changed behavior of crawl stacker
- final flush only when tabletype = RAM
- prestacker (dns prefetch) only if tabletype = RAM and busytime <= 100
- number of maximun entries in stacker is configurable in yacy.init (stacker.slots)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4186 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/PerformanceQueues_p.java, htroot/Status.java, htroot/xml/queues_p.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaWordIndex.java, yacy.init
Mon Oct 29 02:43:20 CET 2007
by orbiter
preparations for mass remote crawls:
two main changes must be implemented to enable mass remote crawls:
- shift control of robots.txt to crawl queue (away from stacker). This is necessary since remote
  crawls can contain unchecked urls. Each peer must check the robots to prevent that it is misused
  as crawl agent for unwanted file retrieval
- implement new index files that control double-check of remotely crawled urls

After removal of robots.txt checking from stacker threads, the multi-threading of this process is void.
Multithreading has been removed. Also the thread pools for the crawl threads had been removed, since
creation of these threads is not resource-consuming, for a detailed explanation see svn 4106

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4181 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlProfileEditor_p.java, htroot/CrawlURLFetchStack_p.java, htroot/CrawlURLFetch_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.html, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/Status.java, htroot/ViewFile.java, htroot/WatchCrawler_p.java, htroot/xml/queues_p.java, htroot/yacy/crawlOrder.java, htroot/yacy/crawlReceipt.java, htroot/yacy/urls.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/plasma/cache/ftp/ResourceInfo.java, source/de/anomic/plasma/crawler/plasmaCrawlQueues.java, source/de/anomic/plasma/crawler/plasmaFTPLoader.java, source/de/anomic/plasma/crawler/plasmaHTTPLoader.java, source/de/anomic/plasma/crawler/plasmaProtocolLoader.java, source/de/anomic/plasma/dbImport/plasmaCrawlNURLImporter.java, source/de/anomic/plasma/plasmaCrawlEntry.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/plasma/plasmaHTCache.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/server/serverDomains.java, source/de/anomic/server/serverSemaphore.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacyURL.java, source/yacy.java, yacy.init
Wed Oct 24 23:38:19 CEST 2007
by fuchsi
* Complete number localization and provide a more reasonable interface to serverObjects:
- put(key, value) methods are now used if a value added to the map should be kept as it is. Numbers are transformed (but not formatted) to an equivalent String representation.
- putASIS(...) have been removed, now done with simple put(...) (see above).
- puNum(...) can be used for number values which should be stored in a formatted way, either depending on the current locale setting for yacy (default) or in a "none" locale (see javadocs and setLocalize()).
- putHTML(...) escapes special characters into corresponding HTML enities ('<' => '<') which was done with put(...) before and so was called too often, becauses it is necessary only for very few cases. Additionally there is a "forXML" mode which only replaces < > & ".
In short: Use put(...) for almost everything, use putXY(...) if you need some special transformation of the value.
A few bugs have been fixed as well, and there should be a small performance improvement for complex pages with a lot of values.

* added additional Sum/Avg rows to access tracker pages, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=456
* removed duplicate code (mostly related to the big changes above).

TODO:
- make sure, number formats work as expected _everywhere_, report overseen stuff http://forum.yacy-websuche.de/viewtopic.php?f=5&t=437
- probably a good idea to add special putDate() methods as they are used in many pages and create duplicated formatting code + maybe some centralized handling for memory value formatting.
- further improve the speed of page creation for the WatchCrawler.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4178 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.html, htroot/AccessTracker_p.java, htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSkins_p.java, htroot/ConfigUpdate_p.java, htroot/Config_p.java, htroot/Connections_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CookieTest.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStartExpert_p.java, htroot/CrawlStartSimple_p.java, htroot/CrawlURLFetchStack_p.java, htroot/CrawlURLFetch_p.java, htroot/FeedReader_p.java, htroot/IndexCleaner_p.java, htroot/IndexControl_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/IndexImport_p.java, htroot/IndexShare_p.java, htroot/IndexTransfer_p.java, htroot/LogStatistics_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.java, htroot/PerformanceSearch_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/Ranking_p.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Statistics.java, htroot/Status.java, htroot/Steering.java, htroot/Supporter.java, htroot/Surftips.java, htroot/TestApplet.java, htroot/User.java, htroot/User_p.java, htroot/ViewFile.html, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/Wiki.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/htdocsdefault/dir.java, htroot/index.java, htroot/js/WatchCrawler.js, htroot/opensearchdescription.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/www/welcome.java, htroot/xml/blacklists_p.java, htroot/xml/bookmarks/posts/all.java, htroot/xml/bookmarks/posts/delete_p.java, htroot/xml/bookmarks/posts/get.java, htroot/xml/bookmarks/tags/get.java, htroot/xml/bookmarks/tags/rename_p.java, htroot/xml/config_p.java, htroot/xml/queues_p.java, htroot/xml/status_p.java, htroot/xml/status_p.xml, htroot/xml/util/getpageinfo_p.java, htroot/yacy/crawlOrder.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/data/htmlTools.java, source/de/anomic/http/httpd.java, source/de/anomic/server/serverMemory.java, source/de/anomic/server/serverObjects.java, source/de/anomic/yacy/yacyClient.java
Mon Oct 22 17:26:47 CEST 2007
by orbiter
distinct keepOrder strategy, more discrete implementation of enhancement introduced in SVN 4158

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4176 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/dbtest.java, source/de/anomic/index/indexContainer.java, source/de/anomic/kelondro/kelondroBytesIntMap.java, source/de/anomic/kelondro/kelondroCache.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/kelondro/kelondroDyn.java, source/de/anomic/kelondro/kelondroFlexSplitTable.java, source/de/anomic/kelondro/kelondroFlexTable.java, source/de/anomic/kelondro/kelondroIndex.java, source/de/anomic/kelondro/kelondroIntBytesMap.java, source/de/anomic/kelondro/kelondroMapTable.java, source/de/anomic/kelondro/kelondroRowSet.java, source/de/anomic/kelondro/kelondroSQLTable.java, source/de/anomic/kelondro/kelondroSplittedTree.java, source/de/anomic/kelondro/kelondroTree.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/yacy/yacyNewsDB.java
Fri Oct 19 06:13:46 CEST 2007
by fuchsi
- Changed yFormatter handling to be more flexible and produce more readable code for server pages. There are serverObject.putNum() methods to allow adding of number type values in a formatted form, and put() methods for number types that add them without formatting. This reduces the need to transform them into Strings in server pages and removes the HTML encoding step which is unecessary for numbers.
- some minor code cleanups (mostly unnecessary casts, null checks)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4166 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/ConfigBasic.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/CrawlStartExpert_p.java, htroot/CrawlStartSimple_p.java, htroot/IndexControl_p.java, htroot/Network.java, htroot/PerformanceMemory_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/WebStructurePicture_p.java, htroot/htdocsdefault/dir.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/kelondro/kelondroBufferedRA.java, source/de/anomic/kelondro/kelondroRow.java, source/de/anomic/plasma/plasmaCondenser.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/server/logging/serverLog.java, source/de/anomic/server/serverCodings.java, source/de/anomic/server/serverObjects.java, source/de/anomic/tools/yFormatter.java, source/de/anomic/ymage/ymageBMPParser.java, source/de/anomic/ymage/ymageCaptcha.java
Tue Oct 16 04:12:31 CEST 2007
by fuchsi
Add a generalized Formatter class yFormatter inspired by http://forum.yacy-websuche.de/viewtopic.php?f=5&t=437
At the current state it allows formatting of numbers (integer + decimal types) for output according to the Locale derived from the language setting in yacy. Network.(html|xml) and Status.html have been changed to use it for now (TODO: should be integrated into other servlets as well to reduce duplicate formatting code).
NOTE: For now the output format for Network.xml simulates the old behaviour which is wrong (it uses '.' as decimal and grouping separator), to make sure external scripts like the yacystats.de one won't break with this update.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4162 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.java, htroot/Status.java, source/de/anomic/data/translator.java, source/de/anomic/tools/yFormatter.java, source/yacy.java
Wed Oct 03 17:06:12 CEST 2007
by orbiter
extended client connection clean-up:
there are now two time-outs, one for the complete connection time, and one for an idle time
connections that are idle for more than 2 minutes are closed, and connections that are alive since more than one hour are also closed
if the complete number of connections exceeds 64, all connections more than 64 and have most idle time are also closed

During normal operation of peers these forced closings should never appear,
but the existence of the idle connection check ensures the availability of the peer and the usability of the host.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4134 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Connections_p.html, htroot/Connections_p.java, source/de/anomic/http/httpc.java, source/de/anomic/server/serverFileUtils.java
Mon Oct 01 14:30:23 CEST 2007
by orbiter
re-designed some parts of DHT position calculation (effect is the same as before)
and replaced old fist hash computation by new method that tries to find a gap in the current dht
to do this, it is necessary that the network bootstraping is done before the own hash is computed
this made further redesigns in peer initialization order necessary

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4117 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Blacklist_p.java, htroot/Blog.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/ConfigBasic.java, htroot/ConfigNetwork_p.java, htroot/ConfigRobotsTxt_p.java, htroot/Connections_p.java, htroot/CrawlStartSimple_p.java, htroot/CrawlURLFetch_p.java, htroot/IndexControl_p.java, htroot/IndexTransfer_p.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Network.java, htroot/News.java, htroot/QuickCrawlLink_p.java, htroot/ScreenSaver.java, htroot/SettingsAck_p.java, htroot/Status.java, htroot/ViewProfile.java, htroot/WatchCrawler_p.java, htroot/Wiki.java, htroot/YaCySearchPluginFF.java, htroot/htdocsdefault/dir.java, htroot/index.java, htroot/opensearchdescription.java, htroot/www/welcome.java, htroot/xml/status_p.java, htroot/yacy/crawlOrder.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transfer.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/wiki/abstractWikiParser.java, source/de/anomic/data/wiki/knwikiParser.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/crawler/AbstractCrawlWorker.java, source/de/anomic/plasma/plasmaCrawlEntry.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaCrawlZURL.java, source/de/anomic/plasma/plasmaDHTChunk.java, source/de/anomic/plasma/plasmaGrafics.java, source/de/anomic/plasma/plasmaRankingDistribution.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSnippetCache.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/server/portForwarding/sch/serverPortForwardingSch.java, source/de/anomic/server/serverCore.java, source/de/anomic/urlRedirector/urlRedirectord.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyDHTAction.java, source/de/anomic/yacy/yacyNetwork.java, source/de/anomic/yacy/yacyNewsRecord.java, source/de/anomic/yacy/yacyPeerActions.java, source/de/anomic/yacy/yacySearch.java, source/de/anomic/yacy/yacySeed.java, source/de/anomic/yacy/yacySeedDB.java
Fri Sep 28 03:21:31 CEST 2007
by orbiter
- redesigned crawl start menu, integrated monitoring pages
- removed web structure picture from indexing menu and grouped it together with htcache monitor
- added a database for terminated crawls, when a crawl is finished it is automatically moved to the new database
- extended crawl profile edit servlet, shows now also terminated crawls
- option that was used to delete profiles is now redesigned to a function that moves the current crawl to the terminated crawls and removes all urls from the current queues!
- fixed here and there problems with indexing queues
- enhances indexing speed by changing cache flush sizes.
- changed behaviour of crawl result servlet: the list of crawled urls is shown if there is one, othevise the overview window is shown

attention: the new profile databases are not compatible with the old one. current crawls will be lost! the web index is not touched.
next steps: the database of terminated crawls can be used to start with them a new crawl. This is useful if one wants to re-crawl specific pages and wants to use a old crawl profile.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4113 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/CacheAdmin_p.html, htroot/CrawlProfileEditor_p.html, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlURLFetch_p.java, htroot/IndexCreateIndexingQueue_p.html, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateLoaderQueue_p.html, htroot/IndexCreateWWWGlobalQueue_p.html, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.html, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.html, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/QuickCrawlLink_p.java, htroot/WatchCrawler_p.html, htroot/WatchCrawler_p.java, htroot/WatchWebStructure_p.html, htroot/WatchWebStructure_p.java, htroot/env/templates/header.template, htroot/env/templates/submenuIndexCreate.template, htroot/env/templates/submenuWebStructure.template, source/de/anomic/data/SitemapParser.java, source/de/anomic/http/httpTemplate.java, source/de/anomic/plasma/crawler/http/CrawlWorker.java, source/de/anomic/plasma/dbImport/SitemapImporter.java, source/de/anomic/plasma/dbImport/plasmaCrawlNURLImporter.java, source/de/anomic/plasma/plasmaCrawlBalancer.java, source/de/anomic/plasma/plasmaCrawlEntry.java, source/de/anomic/plasma/plasmaCrawlNURL.java, source/de/anomic/plasma/plasmaCrawlProfile.java, source/de/anomic/plasma/plasmaCrawlStacker.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/plasmaSwitchboardQueue.java, source/de/anomic/plasma/plasmaWordIndex.java, source/de/anomic/urlRedirector/urlRedirectord.java
Wed Sep 26 12:11:50 CEST 2007
by orbiter
steps to enhance remote search performance:
- added a file size limitation, that disallows parsing of large documents during (offline-) remote search
- added profiling information to search result computation, visible at search access tracker. this info shows used time for URL fetch and snippet computation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4112 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/AccessTracker_p.html, htroot/AccessTracker_p.java, htroot/AccessTracker_p.xml, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/plasma/plasmaSearchEvent.java, source/de/anomic/plasma/plasmaSearchQuery.java, source/de/anomic/plasma/plasmaSnippetCache.java
Tue Sep 25 23:36:08 CEST 2007
by orbiter
- fixed non-closing client connections
- added client connection tracker in connections servelet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4108 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Connections_p.html, htroot/Connections_p.java, htroot/CrawlURLFetch_p.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/robotsParser.java, source/de/anomic/http/httpc.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/crawler/http/CrawlWorker.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Sun Sep 23 22:49:52 CEST 2007
by orbiter
cleanup of unmaintained and outdated performance methods:
removed object pools in httpc. Object pooling is not recommended,
if the creation of the object is not time-intensive. Object pools are only useful,
if there is much computation necessary to create some basic data that is stored
in the object pool and can be re-used. This does not apply to object pools in YaCy.
Object pooling of client sessions would make sense if they would allow re-use of
living connections to other yacy clients. But every connection is closed after usage
of an object in the client pool, therefore the YaCy server client objects are not such
that hold hardware/network-allocated entities.
See:
http://www.javaperformancetuning.com/news/qotm033.shtml
http://java.sun.com/docs/hotspot/HotSpotFAQ.html#gc_pooling
http://docs.sun.com/source/816-7159-10/pt_chap5.html
http://www.microjava.com/articles/techtalk/recylcle2


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4106 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/CrawlURLFetch_p.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/robotsParser.java, source/de/anomic/http/httpc.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/plasma/crawler/http/CrawlWorker.java, source/de/anomic/plasma/plasmaCrawlLURL.java, source/de/anomic/plasma/plasmaParser.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/yacy/yacySeedDB.java, source/yacy.java
Sun Sep 23 19:31:29 CEST 2007
by orbiter
try to fix connection problem, possible cause for wrong junior status and non-passive passive peers:
the YaCy client treats disconnections during data transmissions as error and discards all data transmitted so far
this did not happen so far until I removed a delay time at the end of the daemon session which prevented this case.
To fix this problem, disconnections during transmissions are not treated as error now, which means that end-of-transmissions
with sudden disconnections are not a cause for peer diconnections any more. To be nice to non-updated peers, the sleep time
at the end of server sessions is also re-enabled.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4105 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: source/de/anomic/http/httpc.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/server/serverFileUtils.java, source/de/anomic/yacy/yacyCore.java
Wed Sep 12 02:42:53 CEST 2007
by orbiter
fix for peer ping problem (implemented a 3-time re-ping); cause for 'Connection reset' still unknown

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4095 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.properties, htroot/MessageSend_p.java, source/de/anomic/plasma/plasmaDHTTransfer.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/yacy/yacyClient.java, source/de/anomic/yacy/yacyCore.java, source/de/anomic/yacy/yacyPeerActions.java, yacy.network.unit
Mon Sep 10 08:20:27 CEST 2007
by fuchsi
various fixes and cleanups for blacklist handling:
1. avoid adding duplicate file name entries in config properties for lists, 
2. correctly merge all path masks from all list files for the same host masks,
3. rewrite helper methods standard java methods for Collection transformations,
4. merged various methods with identical functionality for different Collection implementations into one,
5. minor refactoring to improve code readability.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4087 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/Bookmarks.java, htroot/IndexControl_p.java, htroot/sharedBlacklist_p.java, htroot/xml/blacklists_p.java, source/de/anomic/data/blogBoard.java, source/de/anomic/data/blogBoardComments.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/listManager.java, source/de/anomic/plasma/plasmaSwitchboard.java, source/de/anomic/plasma/urlPattern/abstractURLPattern.java, source/de/anomic/plasma/urlPattern/plasmaURLPattern.java
Fri Sep 07 13:45:38 CEST 2007
by orbiter
- fixed/re-implemented media search
- fixed search tipps (topwords, now appearing at the bottom of the page)
- added search consequences execution (deletion of bad referenced some time after the search happened)
- added some formatting at network table

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4078 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: htroot/Network.html, htroot/yacy/search.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/de/anomic/plasma/plasmaCrawlNURL.java, source/de/anomic/plasma/plasmaSearchEvent.java
Wed Sep 05 11:01:35 CEST 2007
by orbiter
joined anomic.net.URL, plasmaURL and url hash computation:
search profiling showed, that a major amount of time is wasted by computing url hashes. The computation does an intranet-check, which needs a DNS lookup. This caused that each urlhash computation needed 100-200 milliseconds, which caused remote searches to delay at least 1 second more that necessary. The solution to this problem is to attach a URL hash to the URL data structure, because that means that the url hash value can be filled after retrieval of the URL from the database. The redesign of the url/urlhash management caused a major redesign of many parts of the software. Since some parts had been decided to be given up they had been removed during this change to avoid unnecessary maintenance of unused code.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4074 6c8d7289-2bf4-0310-a012-ef5d649a1542
Changed Files: build.xml, htroot/Blacklist_p.java, htroot/Bookmarks.java, htroot/CacheAdmin_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigSkins_p.java, htroot/ConfigUpdate_p.java, htroot/CrawlResults.java, htroot/CrawlStartSimple_p.html, htroot/CrawlStartSimple_p.java, htroot/CrawlURLFetchStack_p.java, htroot/CrawlURLFetch_p.java, htroot/FeedReader_p.java, htroot/IndexControl_p.java, htroot/IndexCreateIndexingQueue_p.java, htroot/IndexCreateWWWGlobalQueue_p.java, htroot/IndexCreateWWWLocalQueue_p.java, htroot/IndexCreateWWWRemoteQueue_p.java, htroot/QuickCrawlLink_p.java, htroot/Statistics.java, htroot/Status.java, htroot/Supporter.java, htroot/Surftips.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/WatchCrawler_p.java, htroot/WebStructurePicture_p.java, htroot/htdocsdefault/dir.java, htroot/index.java, htroot/sharedBlacklist_p.java, htroot/xml/bookmarks/posts/delete_p.java, htroot/xml/queues_p.java, htroot/xml/util/getpageinfo_p.java, htroot/yacy/crawlOrder.java, htroot/yacy/list.java, htroot/yacy/search.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/data/SitemapParser.java, source/de/anomic/data/URLFetcherStack.java, source/de/anomic/data/URLLicense.java, source/de/anomic/data/bookmarksDB.java, source/de/anomic/data/robotsParser.java, source/de/anomic/data/userDB.java, source/de/anomic/htmlFilter/htmlFilterContentScraper.java, source/de/anomic/htmlFilter/htmlFilterImageEntry.java, source/de/anomic/htmlFilter/htmlFilterInputStream.java, source/de/anomic/htmlFilter/htmlFilterWriter.java, source/de/anomic/http/httpHeader.java, source/de/anomic/http/httpc.java, source/de/anomic/http/httpd.java, source/de/anomic/http/httpdFileHandler.java, source/de/anomic/http/httpdProxyHandler.java, source/de/anomic/icap/icapd.java, source/de/anomic/index/indexURLEntry.java, source/de/anomic/kelondro/kelondroAbstractRecords.java, source/de/anomic/kelondro/kelondroCollectionIndex.java, source/de/anomic/net/natLib.java, source/de/anomic/plasma/cache/IResourceInfo.java, source/de/anomic/plasma/cache/ResourceInfoFactory.java, source/de/anomic/plasma/cache/ftp/ResourceInfo.java, source/de/anomic/plasma/cache/http/ResourceInfo.java, source/de/anomic/plasma/crawler/AbstractCrawlWorker.java, source/de/anomic/plasma/crawler/ftp/CrawlWorker.java, source/de/anomic/plasma/crawler/http