Open Question Wednesday #23
Jan. 28th, 2009 06:05 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
julieandrewsI'm lying in bed thinking, instead of sleeping. Grr. And I got to wondering. So since it also happens to be Wednesday, maybe you all can stop me from wondering. :)
Do online publications, such as e-magazines, generally allow robots to crawl their site and index and cache their stories?
Do online publications, such as e-magazines, generally allow robots to crawl their site and index and cache their stories?