User-agent: * Disallow: /*? Disallow: /emploi/recherche.html Disallow: /js/common.js User-agent: AhrefsBot User-agent: DataForSeoBot User-agent: stepstoneCrawlBot User-agent: AwarioBot User-agent: SERankingBacklinksBot User-agent: SeekportBot User-agent: Dataprovider.com User-agent: CensysInspect User-agent: SEOkicks User-agent: InternetMeasurement User-agent: SpiderLing User-agent: MixrankBot User-agent: MuckRack User-agent: wpbot User-agent: SaaSBrowserBot User-agent: Jooblebot User-agent: Jobrapido User-agent: Thinkbot User-agent: VelenPublicWebCrawler Disallow: /