From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7E4F28DB7E for ; Fri, 1 Aug 2025 19:46:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754077600; cv=none; b=L1U6rbaBxZTBnSPLUbWomXK9JjVf0EJDkv7K75wLouLhPu+8HG+VyakbYCEiGZ+uGx1BvRkSRvfpXyPiFXe0NvqT6iVje3fRY4t1F3bAdW/c7gFmAv8bxLB3ZPH/MoZmjYgjRdWCon8b0l2POJBkYWPki3NmE5/dMEo5BKQDoLw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754077600; c=relaxed/simple; bh=WotAptjwIWAup+48brKZKm/lb2DwC+FYL/Ve3fdr4EA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=VdMj7MsJftHfm7R5QikiAYvyYli/3HDhoEfHc1p7xzI1qjJJw86XJ4H0erYzxo7TLP96HRpZ14v6fkxjj6MI53r+K+SnXMC+K5EkVkmgM4yIKDArtpPNjM6Vk4N3mmX/liQfnUTc8kHYuH20zzmyFK5TNKByTRL1TvkwKhhqmn8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=ZsWiwX9N; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ZsWiwX9N" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description; bh=Cm/GyiO/Ta9fzZLnf5o8m0dB4dhgpF0PCssXeM4MKXs=; b=ZsWiwX9N4OLT2XXeabe/NgXwxE 5VLz3SwSPrB+0UP+98rHTCuI5yv7y+EDti3w4jJjQHVTheNxPAqm4TPyVFm0clqhlKVOpUQeF1ph1 zejqblKTDv3fSPHCpHDRGtsduSkoaFc8D0r1gmSQWfBngEUTbJbq58S73qnXmaPJM6I8zDEaj4cnT qHS5zROQ3MT4oswzm6C8ICgybayyYSNGAVlpzJNjEXe+6gQbcwNPv9VMq5nWp0/cG9kH1mL/Vx7D0 RBRKgvh/Ya//cGvX+b/3dA3/+hMrDDaGUoZpubFtG/jmTSBtmvbKv0pOAkmgN+hd2ITTYhPrl1lQW 3wWJA+ZA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1uhvi9-00000006hrX-0XGS; Fri, 01 Aug 2025 19:46:37 +0000 From: Luis Chamberlain To: Chuck Lever , Daniel Gomez , kdevops@lists.linux.dev Cc: Luis Chamberlain Subject: [PATCH v3 04/11] style: add extensive code formatting checks to make style Date: Fri, 1 Aug 2025 12:46:28 -0700 Message-ID: <20250801194635.1598544-5-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250801194635.1598544-1-mcgrof@kernel.org> References: <20250801194635.1598544-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: kdevops@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: Luis Chamberlain Extend the 'make style' target with comprehensive code formatting and style checks: 1. Add black formatter for Python files - runs 'black .' to check and format all Python files in the repository 2. Add indentation detection for mixed tabs/spaces: - detect_indentation_issues.py: Detects incorrect indentation based on file type (YAML requires spaces, Makefiles need tabs for recipes, Python prefers spaces) - fix_indentation_issues.py: Automatically fixes indentation issues 3. Keep existing checks: - Whitespace issue detection (trailing spaces, missing newlines) - Commit message format validation - Ensure all text files end with newlines The style checks are ordered to run black first as it provides the most comprehensive Python formatting validation. Indentation and whitespace checks default to checking only modified files to avoid being overwhelmed by pre-existing issues. Generated-by: Claude AI Signed-off-by: Luis Chamberlain --- Makefile | 11 +- scripts/detect_indentation_issues.py | 163 +++++++++++++++++++++++++++ scripts/ensure_newlines.py | 75 ++++++++++++ scripts/fix_indentation_issues.py | 152 +++++++++++++++++++++++++ 4 files changed, 399 insertions(+), 2 deletions(-) create mode 100755 scripts/detect_indentation_issues.py create mode 100755 scripts/ensure_newlines.py create mode 100755 scripts/fix_indentation_issues.py diff --git a/Makefile b/Makefile index a8b58eb7cb88..2c72042d57c8 100644 --- a/Makefile +++ b/Makefile @@ -247,10 +247,17 @@ include scripts/ci.Makefile include scripts/archive.Makefile include scripts/defconfig.Makefile +PHONY += fix-whitespace-last-commit +fix-whitespace-last-commit: + $(Q)git diff --name-only --diff-filter=M HEAD~1..HEAD | xargs -r python3 scripts/fix_whitespace_issues.py + PHONY += style style: - $(Q)python3 scripts/detect_whitespace_issues.py - $(Q)python3 scripts/check_commit_format.py + $(Q)if which black > /dev/null ; then black . || true; fi + $(Q)python3 scripts/detect_whitespace_issues.py || true + $(Q)python3 scripts/detect_indentation_issues.py || true + $(Q)python3 scripts/check_commit_format.py || true + $(Q)python3 scripts/ensure_newlines.py || true PHONY += clean clean: diff --git a/scripts/detect_indentation_issues.py b/scripts/detect_indentation_issues.py new file mode 100755 index 000000000000..98783b4cc54e --- /dev/null +++ b/scripts/detect_indentation_issues.py @@ -0,0 +1,163 @@ +#!/usr/bin/env python3 +""" +Detect indentation issues in files - mixed tabs/spaces or incorrect indentation +""" + +import os +import sys +from pathlib import Path + + +def check_file_indentation(file_path): + """Check a single file for indentation issues""" + issues = [] + + try: + with open(file_path, "rb") as f: + content = f.read() + + # Skip binary files + if b"\0" in content: + return issues + + lines = content.decode("utf-8", errors="ignore").splitlines() + + # Determine expected indentation style + uses_tabs = False + uses_spaces = False + + for line in lines[:100]: # Check first 100 lines to determine style + if line.startswith("\t"): + uses_tabs = True + elif line.startswith(" "): + uses_spaces = True + + # Special rules for certain file types + file_ext = Path(file_path).suffix.lower() + is_yaml = file_ext in [".yml", ".yaml"] + is_makefile = "Makefile" in Path(file_path).name or file_ext == ".mk" + is_python = file_ext == ".py" + + # Check each line for issues + for line_num, line in enumerate(lines, 1): + if not line.strip(): # Skip empty lines + continue + + # Get leading whitespace + leading_ws = line[: len(line) - len(line.lstrip())] + + if is_yaml: + # YAML should use spaces only + if "\t" in leading_ws: + issues.append( + f"Line {line_num}: Tab character in YAML file (should use spaces)" + ) + elif is_makefile: + # Makefiles need tabs for recipe lines + # But can use spaces for variable definitions + stripped = line.lstrip() + if stripped and not stripped.startswith("#"): + # Check if this looks like a recipe line (follows a target) + if line_num > 1 and lines[line_num - 2].strip().endswith(":"): + if leading_ws and not leading_ws.startswith("\t"): + issues.append( + f"Line {line_num}: Recipe line should start with tab" + ) + elif is_python: + # Python should use spaces only + if "\t" in leading_ws: + issues.append( + f"Line {line_num}: Tab character in Python file (PEP 8 recommends spaces)" + ) + else: + # For other files, check for mixed indentation + if leading_ws: + has_tabs = "\t" in leading_ws + has_spaces = " " in leading_ws + + if has_tabs and has_spaces: + issues.append( + f"Line {line_num}: Mixed tabs and spaces in indentation" + ) + elif uses_tabs and uses_spaces: + # File uses both styles + if has_tabs and not uses_tabs: + issues.append( + f"Line {line_num}: Tab indentation (file mostly uses spaces)" + ) + elif has_spaces and not uses_spaces: + issues.append( + f"Line {line_num}: Space indentation (file mostly uses tabs)" + ) + + except Exception as e: + issues.append(f"Error reading file: {e}") + + return issues + + +def main(): + """Main function to scan for indentation issues""" + if len(sys.argv) > 1: + paths = sys.argv[1:] + else: + # Default to git tracked files with modifications + import subprocess + + try: + result = subprocess.run( + ["git", "diff", "--name-only"], + capture_output=True, + text=True, + check=True, + ) + paths = result.stdout.strip().split("\n") if result.stdout.strip() else [] + if not paths: + print("No modified files found in git") + return 0 + except subprocess.CalledProcessError: + print("Error: Not in a git repository or git command failed") + return 1 + except FileNotFoundError: + print("Error: git command not found") + return 1 + + total_issues = 0 + files_with_issues = 0 + + for path_str in paths: + if not path_str: + continue + + path = Path(path_str) + if not path.exists() or not path.is_file(): + continue + + # Skip binary files and certain extensions + if path.suffix in [".pyc", ".so", ".o", ".bin", ".jpg", ".png", ".gif", ".ico"]: + continue + + issues = check_file_indentation(path) + if issues: + files_with_issues += 1 + total_issues += len(issues) + print(f"\n{path}:") + for issue in issues: + print(f" ⚠️ {issue}") + + if total_issues > 0: + print( + f"\nSummary: {total_issues} indentation issues found in {files_with_issues} files" + ) + print("\nTo fix these issues:") + print("- Use spaces only in YAML and Python files") + print("- Use tabs for Makefile recipe lines") + print("- Be consistent with indentation style within each file") + return 1 + else: + print("✅ No indentation issues found!") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/ensure_newlines.py b/scripts/ensure_newlines.py new file mode 100755 index 000000000000..969cf32a2d01 --- /dev/null +++ b/scripts/ensure_newlines.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python3 +""" +Ensure all text files end with a newline +""" + +import os +import sys +from pathlib import Path + + +def needs_newline(file_path): + """Check if file needs a newline at the end""" + try: + with open(file_path, "rb") as f: + content = f.read() + if not content: + return False + # Skip binary files + if b"\0" in content: + return False + return not content.endswith(b"\n") + except: + return False + + +def add_newline(file_path): + """Add newline to end of file""" + try: + with open(file_path, "a") as f: + f.write("\n") + return True + except: + return False + + +def main(): + """Main function""" + extensions = [".py", ".yml", ".yaml", ".sh", ".md", ".txt", ".j2", ".cfg"] + filenames = ["Makefile", "Kconfig", "hosts", ".gitignore", "LICENSE"] + + fixed_count = 0 + + for root, dirs, files in os.walk("."): + # Skip hidden directories and common non-source directories + dirs[:] = [ + d + for d in dirs + if not d.startswith(".") and d not in ["__pycache__", "node_modules"] + ] + + for file in files: + file_path = os.path.join(root, file) + + # Check if file should be processed + should_process = False + if any(file.endswith(ext) for ext in extensions): + should_process = True + elif ( + file in filenames + or file.startswith("Makefile") + or file.endswith("Kconfig") + ): + should_process = True + + if should_process and needs_newline(file_path): + if add_newline(file_path): + print(f"Added newline to: {file_path}") + fixed_count += 1 + + print(f"\nFixed {fixed_count} files") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/fix_indentation_issues.py b/scripts/fix_indentation_issues.py new file mode 100755 index 000000000000..42a5ef9de1fc --- /dev/null +++ b/scripts/fix_indentation_issues.py @@ -0,0 +1,152 @@ +#!/usr/bin/env python3 +""" +Fix indentation issues in files - convert between tabs and spaces as appropriate +""" + +import os +import sys +from pathlib import Path + + +def fix_file_indentation(file_path, dry_run=False): + """Fix indentation in a single file""" + fixed = False + + try: + with open(file_path, "rb") as f: + content = f.read() + + # Skip binary files + if b"\0" in content: + return False + + original_content = content + lines = content.decode("utf-8", errors="ignore").splitlines(keepends=True) + + # Determine file type + file_ext = Path(file_path).suffix.lower() + is_yaml = file_ext in [".yml", ".yaml"] + is_makefile = "Makefile" in Path(file_path).name or file_ext == ".mk" + is_python = file_ext == ".py" + + new_lines = [] + + for line_num, line in enumerate(lines, 1): + if not line.strip(): # Keep empty lines as-is + new_lines.append(line) + continue + + # Get leading whitespace + leading_ws = line[: len(line) - len(line.lstrip())] + rest_of_line = line[len(leading_ws) :] + + if is_yaml or is_python: + # Convert any tabs to spaces (4 spaces per tab) + if "\t" in leading_ws: + new_leading = leading_ws.replace("\t", " ") + new_line = new_leading + rest_of_line + new_lines.append(new_line) + if not dry_run: + print(f" ✅ Line {line_num}: Converted tabs to spaces") + fixed = True + else: + new_lines.append(line) + elif is_makefile: + # Check if this is a recipe line + if line_num > 1 and lines[line_num - 2].strip().endswith(":"): + # Recipe line should start with tab + if leading_ws and not leading_ws.startswith("\t"): + # Convert leading spaces to tab + space_count = len(leading_ws) + tab_count = (space_count + 3) // 4 # Round up + new_leading = "\t" * tab_count + new_line = new_leading + rest_of_line + new_lines.append(new_line) + if not dry_run: + print( + f" ✅ Line {line_num}: Converted spaces to tabs (Makefile recipe)" + ) + fixed = True + else: + new_lines.append(line) + else: + new_lines.append(line) + else: + new_lines.append(line) + + if fixed and not dry_run: + new_content = "".join(new_lines).encode("utf-8") + with open(file_path, "wb") as f: + f.write(new_content) + + except Exception as e: + print(f" ❌ Error processing file: {e}") + return False + + return fixed + + +def main(): + """Main function to fix indentation issues""" + import argparse + + parser = argparse.ArgumentParser(description="Fix indentation issues in files") + parser.add_argument( + "paths", nargs="*", help="Files to check (default: all git tracked files)" + ) + parser.add_argument( + "--dry-run", + action="store_true", + help="Show what would be fixed without changing files", + ) + + args = parser.parse_args() + + if args.paths: + paths = args.paths + else: + # Default to git tracked files + import subprocess + + try: + result = subprocess.run( + ["git", "ls-files"], capture_output=True, text=True, check=True + ) + paths = result.stdout.strip().split("\n") if result.stdout.strip() else [] + except subprocess.CalledProcessError: + print("Error: Not in a git repository") + return 1 + + total_fixed = 0 + + for path_str in paths: + if not path_str: + continue + + path = Path(path_str) + if not path.exists() or not path.is_file(): + continue + + # Skip binary files and certain extensions + if path.suffix in [".pyc", ".so", ".o", ".bin", ".jpg", ".png", ".gif", ".ico"]: + continue + + if fix_file_indentation(path, args.dry_run): + total_fixed += 1 + if not args.dry_run: + print(f"{path}:") + + if total_fixed > 0: + if args.dry_run: + print(f"\nWould fix indentation in {total_fixed} files") + print("Run without --dry-run to apply fixes") + else: + print(f"\n✅ Fixed indentation in {total_fixed} files") + else: + print("✅ No indentation issues found!") + + return 0 + + +if __name__ == "__main__": + sys.exit(main()) -- 2.47.2