User-agent: gsa-crawler Disallow: /comments/ Disallow: /mobile/comments/ Disallow: /report_abuse/ Disallow: /admin_tools/ Disallow: /aba07journ/ Disallow: /mobile/search_results/ Disallow: /abajournal_utility_library/ User-agent: gsa-crawler-aba Disallow: /comments/ Disallow: /mobile/comments/ Disallow: /report_abuse/ Disallow: /admin_tools/ Disallow: /aba07journ/ Disallow: /mobile/search_results/ Disallow: /abajournal_utility_library/ User-agent: EC2LinkFinder Disallow: / ## DEFAULT RULES User-agent: * Crawl-delay: 10 Disallow: /staging/ Disallow: /staging/west/ Disallow: /aba07journ/ Disallow: /newsletter/ Disallow: /newsletter_new/ Disallow: /.src/ Disallow: /.img/ Disallow: /test/ Disallow: /weekly/ Disallow: /inc-popular-today/ Disallow: /member/ Disallow: /files/ Disallow: /images/ Disallow: /test/ Disallow: /themes/ Disallow: /news/kindlefeed/ Disallow: /mag_full_treatment_test/ Disallow: /iphone/ Disallow: /ipad/ Disallow: /admin/ Disallow: /report_abuse/ Disallow: /comments/ Disallow: /mobile/comments/ Disallow: /mobile/search_results/ Disallow: /legalrebelsprofiles/ Disallow: /legalrebelsconnected/ Disallow: /legalrebelsmanifesto/ Disallow: /legalrebelsposts/ Disallow: /legalrebels2009/ Disallow: /legalrebelsresources/ Disallow: /admin_tools/ Disallow: /login/ Disallow: /abajournal_utility_library/