Adding arbitrary rules

[marktran]

Is it possible to add arbitrary rules? e.g. Crawl-delay:

User-agent: msnbot
Crawl-delay: 1

http://www.bing.com/blogs/site_blogs/b/webmaster/archive/2009/08/10/crawl-delay-and-the-bing-crawler-msnbot.aspx

[PikachuEXE]

user_agent :msnbot do
  add_rule 'Crawl-delay', 1
end

According to https://github.com/lassebunk/human_power/blob/master/lib/human_power/generator.rb#L27

Edit: this issue can be changed to request more doc on adding arbitrary rules

[lassebunk]

That's a good idea. I have an idea for how this could be implemented.
Thanks 👍

[PikachuEXE]

Isn't this "implemented"?

[lassebunk]

Not yet. Still thinking 😊

[PikachuEXE]

But #4 (comment) works already?

[lassebunk]

Yes, it will work.
But I'd like for you to be able to write "crawl_delay 1" (shorthand method)

[lassebunk]

Do you know of any other rules that should be added?

[PikachuEXE]

I have no idea :p
@marktran you have any?

[marktran]

That's actually the only one I know of

[lassebunk]

Ok, thanks. I'll keep this in mind. Keeping this open.