neil/talks
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
7859238d09
talks/2024-08-14_boston-python-presentation-night_rockylinux/index.md
Neil Hanlon 7859238d09
sleep helps
2024-08-14 11:18:10 -04:00

19 KiB
Raw Blame History

theme class header footer
default
lead
invert
Python and Rocky Linux Rocky Enterprise Software Foundation

Python and Rocky Linux

(a love story in three parts)

Who am I?

bg right:40%

"Professionally"

  • Rocky Linux cofounder & infra lead
  • Work @ CIQ in OSPO
  • Fedora/EPEL Packager/contributor
  • OpenStack-Ansible reviewer/contributor
  • Menace to society (IPv6 Zealot)

"Unprofessionally"

  • Plays guitar and trumpet
  • Tinkerer, HAM (KC1UYE)
  • pretend electrician

What is Rocky Linux?

Rocky Linux is a community-driven Enterprise Linux distribution--stable enough for the largest enterprise to rely on it, and community-driven to ensureit stays accessible to all

Enterprise Linux?

  • CentOS Linux 8 EOL-ed early; focus shifted to CentOS Stream
  • Built from minimal subset of Fedora package set
  • New major release every 3 years (about six Fedora releases)
  • Each major release supported for 10 years
    • First half: Full support
    • Second half: 'Maintenance' support
  • Community-supported addon repositories like EPEL and rpmfusion
    • Many Python packages not included in the base OS

🐍

Part I - The Operating System

  • platform-python
  • dnf / rpm
  • anaconda (no, not that anaconda)
  • cockpit-project

Cockpit

  • Web-based graphical interface for servers
  • Built with Python and JavaScript
  • Available for Debian and Fedora
  • Think "Webmin but less bad"
  • Manages networking, storage, logs, containers, VMs, etc.

Anaconda (pyanaconda)

  • Installation program for Fedora, Red Hat, others
  • GUI (VNC) or TUI based interaction
  • Supports kickstart files for unattended installations
  • Uses Gtk.Builder / Glade for customization
  • UI being rewritten for fewer rough edges
    • Expected in Fedora 42
  • Blivet: Storage library for disk setup
  • Addons: ostree, openscap, etc.

DNF ("Dandified Yum")

  • Package manager for Fedora, RHEL, SUSE, and others
  • Rewritten version of yum, which was a rewrite of yup
  • Layer on top of rpm for package management
  • DNF4 (2013): Python + C/C++
    • We don't talk about DNF 1-3...
  • DNF5 (2018): C++
    • Currently only in Fedora

DNF4 and Python

  • Lots of probably boring history
  • External dependency solver: faster, less memory usage
  • Python 2/3 transition was (and is) challenging
  • libdnf: High-level API for DNF (C/C++)
  • libsolv: Package satisfiability solver (C)
  • librepo: libcURL wrapper for fetching repo metadata and packages (C/Python bindings)
  • libcomps: Alternative to slower python yum.comps library (C/Python bindings)

Platform-Python

  • System must have a stable Python version
  • Rocky 8: Python 3.6.8
  • Rocky 9: Python 3.9.18
  • System python always located at /usr/libexec/platform-python

Python on EL - Pitfalls

  • Tools like Ansible may get confused
    • May need to force ansible_interpreter
  • Many Python packages/modules built against specific Python versions
  • RPM modularity exacerbates the issue
  • Many end up using pip for package installation :(
  • No solid solution yet

Using / Developing with Python on Rocky

How Hard Is It?

  • It depends!
  • More challenging and frustrating than ideal
  • Use pkgs.org for finding non-standard repos (EPEL, rpmfusion)
  • Building RPMs isn't too hard, but pip is often your best friend

Python on Rocky

Rocky 8

  • Modular and non-modular Python versions
  • Platform-python (non-modular) required for dnf
  • Most packages/modules built against default Python
  • Modular python not modularized "correctly"

Rocky 9

  • Platform-python == Python3
    • Required for dnf
  • Python provided as separate packages (python3.11, python3.12)
    • Installable in parallel

Part B - Modularity

Really Quick Backstory, I Promise

  • Before: Carefully managed servers like a backyard garden
  • Now: Fully industrialized farm with automated preparation, planting, and harvesting
  • Cattle vs. Pets analogy
  • Development cycles now measured in days
  • EL releases need to be supported for 10 years, but new tech evolves rapidly (e.g., new JS frameworks)
  • Diverse tech stack: Python, Postgres, Node.js, Nginx, PHP, Ruby (some evolving faster than others)
  • "It is easier to move a problem around than it is to solve it."
  • What if users could compose their "own" OS?

bg

So We're Doomed?

  • Can't package everything
  • Can maintain the important stuff
  • Software velocity isn't slowing down
  • We humans are only so fast
  • Focus on improving tooling

Part 3 - Empanadas and friends

Building Rocky Linux

  • Leveraging Fedora tools: koji, mock, MBS, pungi, lorax
  • Distrobuild: enhances koji/MBS control
  • PV2: used for automating upstream imports
  • Recognizing our challenges: "It's us, we're the problem"
  • Apollo: errata publisher requiring improvements

empanadas (git.resf.org/sig_core/toolkit)

  • empandas is the way Rocky does release engineering and image building
  • toolkit began as a bunch of (bash) scripts
    • began compiling into empanadas as a python CLI
  • click, rpm python, ... subprocess.run()
    • like an embarrassing amount of subprocess.run()
  • I don't claim to be a good Python dev
  • some refactoring efforts going on

Friends

  • mirrormanager
  • mailman/hyperkitty
  • kiwi
  • others I've surely forgotten

What does it all mean?

  • Python is core to Fedora, CentOS, and Rocky Linux distributions
  • Tremendous amount of Python used in building and distributing the OS
  • Python is approachable for "sysadmin-adjacent" roles, even if the code isn’t always Pythonic
  • RFC-1925 always has an answer to any question

Thank You!

Q & A

Join us on Mattermost! https://chat.rockylinux.org