CyberRota Analysis
AI-GeneratedThe vulnerability affects the Protego robots.txt parser, specifically in its handling of asterisks in Allow or Disallow directives, which can lead to exponential backtracking and potential denial of service when processing crafted robots.txt files. This could allow an attacker to manipulate the parser's behavior, causing it to consume excessive resources. Organizations using versions prior to 0.6.2 should prioritize updating to mitigate the risk of service disruption.
Public Exploit Signal
A public exploit, PoC, GitHub repository or Metasploit reference was detected for this CVE.
Note: these links are listed for security research and verification purposes only.
Original NVD Description
Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego._urlpattern._URLPattern._prepare_pattern_for_regex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks creates exponential backtracking. After protego.Protego.parse processes a crafted robots.txt file, protego.Protego.can_fetch can spend an attacker-controlled period matching a near-miss URL and deny service to the crawler. The vulnerable path is src/protego/_urlpattern.py in the _URLPattern match logic. This issue is fixed in version 0.6.2.