From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01829259C80 for ; Fri, 11 Apr 2025 18:23:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.16 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744395797; cv=none; b=BKTCVD1/DnJZZ6xHBq+zGP9NEWWG59WFB8Y045yDlt+VsQDzuRB4QnQhr/4Ln8HcstF2zIZH7STGF/G92as9mDSofJs8+qNTFHrpK/DidPNud4HwRvf6+8wCx9Zux5xNpkm4A2tgahW4Hlo5eh4lUUWfT//PPsBSfrxyvkYabZY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744395797; c=relaxed/simple; bh=gy3PH4NlMwaJrzBBR7rwvtuNGrtUSzQUwhhDHqG8R6o=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pr+XTS8UytgiEICcomh9Qwr/F3uAb+El5HTYQtJB1ORWF6U0GMO+Tpz7w1CxpmEEJjW8c1J5CVy9sZ2kY0RMqE5jlgV1EeHYFbWHzbnSXmn+Jf/+3/CdtevQqAxsJmwKI6BgvNrE6vYkOYU7p2cOSII8/EFb5xRl5DL22Faj6sE= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iNR4LXg5; arc=none smtp.client-ip=192.198.163.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iNR4LXg5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1744395796; x=1775931796; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=gy3PH4NlMwaJrzBBR7rwvtuNGrtUSzQUwhhDHqG8R6o=; b=iNR4LXg5iVJqud4LCJsdUA0FELm+W/gCvZEvFa/3LtG1baT4IjKEBFwX B6guPkfREjFF1+1gLT3Isw7lH6PDLqaAAmOikCajuqjdfpBrtB1C4QRBP XAzyUQA7BYSOZXq5oESaAs9wCyeBVgiuP0GY7RaI3JAgLg6vJPvmjR1SS 0Up3uRiOQLbtqn5jWWIW+lsU+Y+kFZrK6uSy91uHXFxiltWot+ZticTWL 21abzgMoZJ0HBdji1VcET/hnE6u8VfLwtfemkp7wSSPJuLcoVXoTSnglo u/5S2PogFcJhYIlQynFC7irlUER0Kbndd5U5UHPTo+9WXpenXdQJHsqOB g==; X-CSE-ConnectionGUID: 66/s91BET9ydK6/nIdnJnQ== X-CSE-MsgGUID: iKSNE1dRRQyX/rew6Rl2mQ== X-IronPort-AV: E=McAfee;i="6700,10204,11401"; a="33570197" X-IronPort-AV: E=Sophos;i="6.15,205,1739865600"; d="scan'208";a="33570197" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Apr 2025 11:23:14 -0700 X-CSE-ConnectionGUID: eHmKeAF3Rb6Ab3R8uapNZw== X-CSE-MsgGUID: D2dfMZqQQtyvMX6bxjvJMQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.15,205,1739865600"; d="scan'208";a="166459177" Received: from agluck-desk3.sc.intel.com (HELO agluck-desk3) ([172.25.222.70]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Apr 2025 11:23:13 -0700 Date: Fri, 11 Apr 2025 11:23:12 -0700 From: "Luck, Tony" To: Konstantin Ryabitsev Cc: Daniel Gomez , users@kernel.org, Luis Chamberlain , Chuck Lever , kdevops@lists.linux.dev Subject: Re: Forbidden requests for kernel.org/releases.json Message-ID: References: <20250410-prophetic-elegant-fossa-a210fb@lemur> Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250410-prophetic-elegant-fossa-a210fb@lemur> On Thu, Apr 10, 2025 at 08:45:30AM -0400, Konstantin Ryabitsev wrote: > On Thu, Apr 10, 2025 at 10:05:28AM +0200, Daniel Gomez wrote: > > We've started encountering "HTTP Error 403: Forbidden" errors in kdevops > > when querying https://www.kernel.org/releases.json from our CI environments/ > > deployments. We're using a Python script with the urllib library to fetch the > > latest kernel release information [1]. > > Yes, I'm trying to deal with bots who don't identify themselves. We're seeing > many requests per second from user-agents like "python-requests/x.x" or > "Python-urllib/x.x" or "Java/x.x" etc, and it's impossible for us to tell good > bots from bad bots if they don't identify themselves properly. > > > * What is the recommended approach for automated tools to access > > kernel.org/releases.json? > > Set your user-agent to something like: > > "kdevops-ci/{version} (contact@address.here)" Would it help things if you just delayed response to *every* query by a couple of seconds? While I love it that lore.kernel.org answers my queries more or less instantly, my user experience wouldn't suffer significantly if I had to wait for a short (to humans) but long (to bots) time. -Tony