Comments on: Web site harvesting – it’s happening now. http://gm22.net/2006/08/30/web-site-harvesting-%e2%80%93-it%e2%80%99s-happening-now/ What? No lemongrass? Tue, 15 Dec 2009 13:04:46 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: nev http://gm22.net/2006/08/30/web-site-harvesting-%e2%80%93-it%e2%80%99s-happening-now/comment-page-1/#comment-4 nev Thu, 07 Sep 2006 03:52:04 +0000 http://gm22.net/2006/08/30/web-site-harvesting-%e2%80%93-it%e2%80%99s-happening-now/#comment-4 Hi G, This is nothing new. What about all those Search Engines indexing the web tying up bandwidth ? How many of those #$#% things does the world need ? One educational facility I know of is using the Google Search. It indexes about 800,000 pages everytime. Imagine the cost of that bandwidth. Not a bad argument for a Google Appliance. And what of the old Time Machine ( www.archive.org ). They've been doing it for years. Advertisements? Big bandwidth hogs... We actually pay to receive advertising. What a bunch of suckers we are !!! I'd love to see ISP's block ads altogether to preserve our precious bandwidth. I don't block Flash for nothing you know... Really needs the whole population to wise-up... OR get on the wagon ourselves :) Like archive.org, at least the nla does it for an admirable reason :) Consider it a history tax, Mr Ripper :) my 0.02 cents :) Hi G,

This is nothing new.

What about all those Search Engines indexing the web tying up bandwidth ?

How many of those #$#% things does the world need ?

One educational facility I know of is using the Google Search. It indexes about 800,000 pages everytime. Imagine the cost of that bandwidth. Not a bad argument for a Google Appliance.

And what of the old Time Machine ( http://www.archive.org ). They’ve been doing it for years.

Advertisements? Big bandwidth hogs… We actually pay to receive advertising. What a bunch of suckers we are !!! I’d love to see ISP’s block ads altogether to preserve our precious bandwidth.

I don’t block Flash for nothing you know…

Really needs the whole population to wise-up…

OR get on the wagon ourselves :)

Like archive.org, at least the nla does it for an admirable reason :)

Consider it a history tax, Mr Ripper :)

my 0.02 cents :)

]]> By: Meg http://gm22.net/2006/08/30/web-site-harvesting-%e2%80%93-it%e2%80%99s-happening-now/comment-page-1/#comment-3 Meg Wed, 30 Aug 2006 11:15:09 +0000 http://gm22.net/2006/08/30/web-site-harvesting-%e2%80%93-it%e2%80%99s-happening-now/#comment-3 It's really not any different to the way the Internet Archive crawls the net, and I don't think that privacy laws are really applicable to content that's freely available online anyway. Perhaps you're thinking of copyright laws? Anyway, the process that's normally followed by Pandora is: 1. Site identified as 'significant' (Guidelines from the State Library of WA for this are online here: http://www.slwa.wa.gov.au/pdf/pandoraoct02.pdf) by one of several archiving libraries in Australia. 2. Site owner is contacted for permission to archive (I'm not sure what happens if they are unable to find contact information on the owner). 3. Site submitted to Pandora and indexed. This crawl (and the previous one in 2005) are different in that they are crawling all of the *.au namespace (and sites that are identified as having an Australian IP address). However the content will not become publicly available from Pandora without the site owners permission - they do state that on their information page. If you're ever contacted by the NLA to have your site included in Pandora it's pretty cool to know that your site is considered "of interest" for posterity. It’s really not any different to the way the Internet Archive crawls the net, and I don’t think that privacy laws are really applicable to content that’s freely available online anyway. Perhaps you’re thinking of copyright laws?

Anyway, the process that’s normally followed by Pandora is:

1. Site identified as ’significant’ (Guidelines from the State Library of WA for this are online here:
http://www.slwa.wa.gov.au/pdf/pandoraoct02.pdf)
by one of several archiving libraries in Australia.
2. Site owner is contacted for permission to archive (I’m not sure what happens if they are unable to find contact information on the owner).
3. Site submitted to Pandora and indexed.

This crawl (and the previous one in 2005) are different in that they are crawling all of the *.au namespace (and sites that are identified as having an Australian IP address). However the content will not become publicly available from Pandora without the site owners permission – they do state that on their information page.

If you’re ever contacted by the NLA to have your site included in Pandora it’s pretty cool to know that your site is considered “of interest” for posterity.

]]> By: Tuna http://gm22.net/2006/08/30/web-site-harvesting-%e2%80%93-it%e2%80%99s-happening-now/comment-page-1/#comment-2 Tuna Wed, 30 Aug 2006 05:38:58 +0000 http://gm22.net/2006/08/30/web-site-harvesting-%e2%80%93-it%e2%80%99s-happening-now/#comment-2 Yes but its not all the web! see - http://pandora.nla.gov.au/selectionguidelinesallpartners.html "PANDORA is a selective archive. The National Library and its partners do not attempt to collect all Australian online publications and web sites, but select those that they consider are of significance and to have long-term research value." You can also opt out. Some Govt Agencies opt out for legal reasons. Yes but its not all the web! see –

http://pandora.nla.gov.au/selectionguidelinesallpartners.html

“PANDORA is a selective archive. The National Library and its partners do not attempt to collect all Australian online publications and web sites, but select those that they consider are of significance and to have long-term research value.”

You can also opt out. Some Govt Agencies opt out for legal reasons.

]]>