Funferal

Robots.txt and irlgov.ie

Over at Gavin’s Blog there’s an interesting observation about the robots.txt file at www.irlgov.ie. Now, robots.txt files are not something I’m given to browsing on an ongoing basis, but there do seem to be some strange items on the ignore list – specifically, several documents that no longer appear on the websites of individual state bodies, such as missing documents from the Department of Foreign Affairs’ press office. More interestingly, an entire day of files from the hearings of the Committee of Public Accounts. The items covered on that day are:

Financial Statements 1994-97: Rent Tribunal
Supplementary Welfare Allowances:
Value for Money Examination Department of Social, Community and Family Affairs (Resumed)

Wonder why they went missing?