YAML Bulletproofing Proposal: Making packages.yaml 100% Reliable
Executive Summary
This proposal outlines a comprehensive plan to bulletproof the existing packages.yaml configuration system. Rather than replacing YAML (which is the right choice), we’ll add robust validation, safety checks, and error recovery to eliminate potential failure points.
Current State Analysis
Strengths âś…
- Well-Structured: Clear hierarchical organization (core → system → desktop → development → media)
- Consistent Format: All modules follow the same pattern (packages, configs, commands, depends, start, end, type)
- Working Validation: Basic field validation exists in
source/config/loader.go - Dependency System: Modules can depend on other modules
Vulnerabilities 🚨
- No Cycle Detection: Circular dependencies could cause infinite loops
- Unsafe Commands: Raw shell commands without safety validation
- Missing Package Validation: No check if packages exist before installation
- Limited Schema Validation: Only basic field presence checks
- No Error Recovery: Single failure can break entire installation
Bulletproofing Strategy
Phase 1: Critical Safety (HIGH PRIORITY)
1.1 Dependency Cycle Detection
Problem: Module A depends on B, B depends on C, C depends on A = infinite loop Solution: Graph traversal algorithm to detect cycles before execution
// NEW: source/config/dependency_validator.go
func ValidateDependencies(cfg *Config) error {
allModules := collectAllModules(cfg)
// Check for missing dependencies
for moduleID, module := range allModules {
for _, dep := range module.Depends {
if _, exists := allModules[dep]; !exists {
return fmt.Errorf("module %s depends on non-existent module %s", moduleID, dep)
}
}
}
// Check for circular dependencies using DFS
for moduleID := range allModules {
if hasCyclicDependency(moduleID, allModules, make(map[string]bool), make(map[string]bool)) {
return fmt.Errorf("circular dependency detected involving module %s", moduleID)
}
}
return nil
}
Benefits:
- Prevents installation deadlocks
- Clear error messages for dependency issues
- Validates entire dependency graph before starting
1.2 Command Safety Validation
Problem: Commands like rm -rf / or chmod 777 could be catastrophic
Solution: Pattern-based command validation with safety requirements
// NEW: source/config/command_validator.go
var dangerousPatterns = []*regexp.Regexp{
regexp.MustCompile(`rm\s+-rf\s+/[^h]`), // Never rm -rf / (except /home/user subdirs)
regexp.MustCompile(`>\s*/dev/sd[a-z]`), // Don't write to block devices
regexp.MustCompile(`dd\s+.*of=/dev/`), // Dangerous dd commands
regexp.MustCompile(`chmod\s+777`), // Overly permissive permissions
regexp.MustCompile(`\$\(.*curl.*\|.*sh\)`), // Pipe curl to shell
regexp.MustCompile(`sudo\s+rm\s+-rf\s+/`), // Sudo + dangerous rm
}
func ValidateCommands(module Module, moduleID string) error {
for i, cmd := range module.Commands {
// Check for dangerous patterns
for _, pattern := range dangerousPatterns {
if pattern.MatchString(cmd) {
return fmt.Errorf("module %s command %d contains dangerous pattern: %s",
moduleID, i+1, cmd)
}
}
// Require safety patterns for critical operations
if strings.Contains(cmd, "sudo") && !strings.Contains(cmd, "|| true") && !isSafeCommand(cmd) {
return fmt.Errorf("module %s command %d uses sudo but lacks safety handling: %s",
moduleID, i+1, cmd)
}
}
return nil
}
Benefits:
- Prevents accidental system destruction
-
Enforces “  true” pattern for non-critical sudo commands - Blocks known dangerous command patterns
1.3 JSON Schema Validation
Problem: YAML structure errors only caught at runtime Solution: Comprehensive schema validation with clear error messages
// NEW: install/packages.schema.json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "ArchRiot Package Configuration",
"type": "object",
"properties": {
"metadata": {
"type": "object",
"properties": {
"schema_version": { "type": "string" },
"archriot_version": { "type": "string" },
"arch_required": { "enum": ["x86_64", "aarch64", "any"] }
},
"required": ["schema_version"]
},
"core": { "$ref": "#/$defs/moduleCategory" },
"desktop": { "$ref": "#/$defs/moduleCategory" },
"system": { "$ref": "#/$defs/moduleCategory" },
"development": { "$ref": "#/$defs/moduleCategory" },
"media": { "$ref": "#/$defs/moduleCategory" }
},
"$defs": {
"moduleCategory": {
"type": "object",
"patternProperties": {
"^[a-z][a-z0-9_]*$": { "$ref": "#/$defs/module" }
},
"additionalProperties": false
},
"module": {
"type": "object",
"required": ["start", "end", "type"],
"properties": {
"packages": {
"type": "array",
"items": {
"type": "string",
"pattern": "^[a-zA-Z0-9._+-]+$",
"minLength": 1
}
},
"configs": {
"type": "array",
"items": {
"type": "object",
"required": ["pattern"],
"properties": {
"pattern": { "type": "string", "minLength": 1 },
"target": { "type": "string" },
"preserve_if_exists": {
"oneOf": [
{ "type": "boolean" },
{ "type": "array", "items": { "type": "string" } }
]
}
}
}
},
"commands": {
"type": "array",
"items": { "type": "string", "minLength": 1 }
},
"depends": {
"type": "array",
"items": {
"type": "string",
"pattern": "^[a-z]+\\.[a-z][a-z0-9_]*$",
"description": "Format: category.module (e.g., core.base)"
}
},
"start": { "type": "string", "minLength": 1 },
"end": { "type": "string", "minLength": 1 },
"type": {
"enum": ["Package", "Git", "System", "File", "Module"]
},
"critical": {
"type": "boolean",
"description": "If true, failure stops entire installation"
},
"retry_policy": {
"type": "object",
"properties": {
"max_attempts": { "type": "integer", "minimum": 1, "maximum": 10 },
"backoff_seconds": { "type": "integer", "minimum": 1, "maximum": 300 }
}
}
},
"additionalProperties": false
}
}
}
Benefits:
- Catches structural errors before execution
- Validates package names, dependency format, command syntax
- Provides clear error messages with line numbers
- IDE integration for real-time validation
Phase 2: Enhanced Reliability (MEDIUM PRIORITY)
2.1 Package Availability Validation
Problem: Installation fails if packages don’t exist in repos Solution: Pre-validate all packages against pacman/AUR
// NEW: source/config/package_validator.go
func ValidatePackageAvailability(packages []string, moduleID string) error {
var missing []string
for _, pkg := range packages {
if !isPackageAvailable(pkg) {
missing = append(missing, pkg)
}
}
if len(missing) > 0 {
return fmt.Errorf("module %s references unavailable packages: %v", moduleID, missing)
}
return nil
}
func isPackageAvailable(pkg string) bool {
// Check official repos first (faster)
if err := exec.Command("pacman", "-Si", pkg).Run(); err == nil {
return true
}
// Check AUR if yay is available
if _, err := exec.LookPath("yay"); err == nil {
if err := exec.Command("yay", "-Si", pkg).Run(); err == nil {
return true
}
}
return false
}
2.2 Enhanced YAML Structure with Metadata
Enhancement: Add metadata section for versioning and compatibility
# NEW: Enhanced packages.yaml structure
metadata:
schema_version: "1.0.0"
archriot_version: ">=2.9.0"
arch_required: "x86_64"
last_updated: "2024-12-19"
validators:
- dependency_cycles
- command_safety
- package_availability
- schema_compliance
core:
base:
packages:
- base-devel
- git
critical: true # Installation fails if this fails
retry_policy:
max_attempts: 3
backoff_seconds: 5
# ... rest of existing structure
2.3 Rollback and Recovery Mechanisms
Enhancement: Add rollback commands for safer installations
desktop:
hyprland:
packages:
- hyprland
- waybar
rollback_commands:
- "sudo pacman -Rns --noconfirm hyprland waybar || true"
- "rm -rf ~/.config/hypr ~/.config/waybar || true"
# If installation fails, run rollback_commands automatically
Phase 3: Advanced Features (LOW PRIORITY)
3.1 Configuration Drift Detection
Feature: Detect when system configs have changed since installation
3.2 Update Compatibility Checking
Feature: Validate that config updates are compatible with installed versions
3.3 Performance Optimization
Feature: Parallel package installation where dependencies allow
Implementation Plan
Week 1: Core Safety
- Implement dependency cycle detection
- Add command safety validation
- Create JSON schema file
- Integrate schema validation into loader
Week 2: Enhanced Validation
- Package availability checking
- Metadata section implementation
- Enhanced error messages
- Comprehensive testing
Week 3: Error Recovery
- Retry mechanisms
- Rollback system
- Critical vs non-critical module handling
- Documentation updates
Expected Benefits
Reliability Improvements
- 99%+ Success Rate: Catch issues before they cause failures
- Zero Infinite Loops: Dependency cycle detection prevents deadlocks
- Safe Commands: Dangerous operations blocked or require explicit override
- Clear Error Messages: Know exactly what failed and why
Maintenance Benefits
- Schema Validation: IDE support with autocomplete and error highlighting
- Automated Testing: Validate config changes in CI/CD
- Version Management: Track compatibility and breaking changes
- Documentation: Self-documenting schema with field descriptions
User Experience
- Faster Failures: Catch issues in validation phase (seconds) vs installation phase (minutes)
- Recovery Options: Automatic rollback on failure
- Progress Visibility: Know if failure is critical or can be skipped
Risk Assessment
Low Risk Changes
- Dependency validation (read-only analysis)
- Schema validation (opt-in initially)
- Enhanced error messages
Medium Risk Changes
- Command validation (could block legitimate commands)
- Package availability checking (network dependent)
Mitigation Strategies
- Gradual Rollout: Implement as warnings first, then errors
- Override Mechanisms: Allow experienced users to bypass validations
- Comprehensive Testing: Test on multiple systems before release
Recommendation
PROCEED WITH PHASE 1 - The critical safety features provide huge reliability improvements with minimal risk. Start with dependency cycle detection as it’s completely safe and catches a real failure mode.
This bulletproofing approach keeps the excellent YAML structure while making it production-grade reliable.