diff --git a/2024-08-14_boston-python-presentation-night_rockylinux/E_rTivEXMAg6Tjh.png b/2024-08-14_boston-python-presentation-night_rockylinux/E_rTivEXMAg6Tjh.png new file mode 100644 index 0000000..dbeed67 Binary files /dev/null and b/2024-08-14_boston-python-presentation-night_rockylinux/E_rTivEXMAg6Tjh.png differ diff --git a/2024-08-14_boston-python-presentation-night_rockylinux/chat.rockylinux.org.png b/2024-08-14_boston-python-presentation-night_rockylinux/chat.rockylinux.org.png new file mode 100644 index 0000000..b1b9b63 Binary files /dev/null and b/2024-08-14_boston-python-presentation-night_rockylinux/chat.rockylinux.org.png differ diff --git a/2024-08-14_boston-python-presentation-night_rockylinux/empanadas.png b/2024-08-14_boston-python-presentation-night_rockylinux/empanadas.png new file mode 100644 index 0000000..6bd8dbe Binary files /dev/null and b/2024-08-14_boston-python-presentation-night_rockylinux/empanadas.png differ diff --git a/2024-08-14_boston-python-presentation-night_rockylinux/index.md b/2024-08-14_boston-python-presentation-night_rockylinux/index.md index 1c40dd2..d52c30d 100644 --- a/2024-08-14_boston-python-presentation-night_rockylinux/index.md +++ b/2024-08-14_boston-python-presentation-night_rockylinux/index.md @@ -8,23 +8,26 @@ footer: Rocky Enterprise Software Foundation --- # Python and Rocky Linux -## (a love story in three parts) +
+

(a love story in three parts)

+
--- + + ## Who am I? ![bg right:40%](bg.png) +
+ ### "Professionally" - Rocky Linux cofounder & infra lead @@ -33,12 +36,21 @@ I'm also a Fedora packager, contributor to OpenStack-Ansible, and in a former li - OpenStack-Ansible reviewer/contributor - Menace to society (IPv6 Zealot) +
+ +
+ ### "Unprofessionally" - Plays guitar and trumpet -- Tinkerer, HAM (KC1UYE), pretend electrician +- Tinkerer, HAM (KC1UYE) +- pretend electrician + +
--- + + # What is Rocky Linux? # Rocky Linux is a community-driven Enterprise Linux distribution--stable enough for the largest enterprise to rely on it, and community-driven to ensureit stays accessible to all --- + + + -# The vision of the Rocky Enterprise Software Foundation is to create and nurture a community of individuals and organizations that are committed to ensuring the longevity, stewardship, and innovation of enterprise-class open-source software +## Enterprise Linux? + +
+ +
+ +* CentOS Linux 8 EOL-ed early; focus shifted to CentOS Stream +* Built from minimal subset of Fedora package set +* New major release every 3 years (about six Fedora releases) +* Each major release supported for 10 years + - First half: Full support + - Second half: 'Maintenance' support +* Community-supported addon repositories like EPEL and rpmfusion + - Many Python packages not included in the base OS --- + + -# Cool.. So what about the python stuff? - - +

:snake:

--- # Part I - The Operating System -(in no particular order)
* platform-python -* dnf +* dnf / rpm * anaconda (no, not that anaconda) * cockpit-project @@ -98,132 +126,173 @@ I'm getting there! Good stories require exposition. --- -## anaconda (pyanaconda) + +## Cockpit +* Web-based graphical interface for servers +* Built with Python and JavaScript +* Available for Debian and Fedora +* Think "Webmin but less bad" +* Manages networking, storage, logs, containers, VMs, etc. + + + +--- - -* installation program for Fedora, Red Hat, others +## Anaconda (pyanaconda) +* Installation program for Fedora, Red Hat, others * GUI (VNC) or TUI based interaction - * can also process kickstart file for unattended installations -* Gtk.Builder / Glade - customization and modification - * I actively avoid this because it's painful - * UI parts being rewritten to have fewer rough edges - - Fedora 42, probably -* blivet - storage library used by anaconda for setting up server's disks -* addons: ostree, openscap, etc +* Supports kickstart files for unattended installations +* Uses Gtk.Builder / Glade for customization +* UI being rewritten for fewer rough edges + - Expected in Fedora 42 +* Blivet: Storage library for disk setup +* Addons: ostree, openscap, etc. + + --- -## cockpit - - -* web-based graphical interface for servers -* python, javascript -* available for Debian as well as Fedora -* think "webmin but less bad" -* networking, storage, logs, containers/VMs, and so on +## DNF ("Dandified Yum") +* Package manager for Fedora, RHEL, SUSE, and others +* Rewritten version of `yum`, which was a rewrite of `yup` +* Layer on top of `rpm` for package management +* DNF4 (2013): Python + C/C++ + - We don't talk about DNF 1-3... +* DNF5 (2018): C++ + - Currently only in Fedora --- -## dnf ("dandified yum") - - -* package manager for Fedora, RHEL, SUSE, some others -* rewrite of `yum` which is a rewrite of `yup`... etc -* layer on top of `rpm` for package management -* dnf4 (2013) - python + C/C++ - - we don't talk about dnf 1-3... -* dnf5 (2018) - C++ - - only in Fedora right now +### DNF4 and Python +* Lots of probably boring history +* External dependency solver: faster, less memory usage +* Python 2/3 transition was (and is) challenging +* libdnf: High-level API for DNF (C/C++) +* libsolv: Package satisfiability solver (C) +* librepo: libcURL wrapper for fetching repo metadata and packages (C/Python bindings) +* libcomps: Alternative to slower python yum.comps library (C/Python bindings) --- + + -### dnf4 and python +## Platform-Python -* lots of probably boring history -* external depsolver -* python 2/3 transition period was (is) a dark time -* libdnf - - high level API for DNF (C/C++) -* libsolv - - package satisfiability solver (C) -* librepo - - libcURL wrapper for fetching Linux repo metadata and packages (C/Python bindings) -* libcomps - - alternative for slower python yum.comps library (C/Python bindings) - ---- - -## platform-python - - - -* system must have a stable python -* rocky 8: 3.6.8 -* rocky 9: 3.9.18 -* /usr/libexec/platform-python +* System must have a stable Python version +* Rocky 8: Python 3.6.8 +* Rocky 9: Python 3.9.18 +* System python always located at `/usr/libexec/platform-python`
-## python on EL - pitfalls - -* tools like ansible are easily confused - - may need to force ansible_interpreter -* many python packages/modules are built against specific pythons -* RPM modularity makes this problem worse -* most people just end up installing things using pip :( - * we don't really have a good solution for this :/ +## Python on EL - Pitfalls +* Tools like Ansible may get confused + - May need to force `ansible_interpreter` +* Many Python packages/modules built against specific Python versions +* RPM modularity exacerbates the issue +* Many end up using pip for package installation :( +* No solid solution yet
--- + + -## using / developing with python on Rocky +## Using / Developing with Python on Rocky -* it depends! -* harder than it could be. way more frustrating than it should be -* pkgs.org can help find things in non-standard repos (EPEL, rpmfusion) -* building RPM packages isn't that difficult, but pip is probably your best friend +
+ +### How Hard Is It? +* It depends! +* More challenging and frustrating than ideal +* Use pkgs.org for finding non-standard repos (EPEL, rpmfusion) +* Building RPMs isn't too hard, but pip is often your best friend + +
+ + --- -### python on Rocky +### Python on Rocky + + #### Rocky 8 - -* modular **and** non-modular python -* platform-python for dnf is non-modular -* most python packages/modules only built against default python - +* Modular and non-modular Python versions +* Platform-python (non-modular) required for dnf +* Most packages/modules built against default Python +* Modular python not modularized "correctly" #### Rocky 9 - -* platform-python == python3 - - required by dnf -* python not provided as modules but as seperately named packages (python3.11, python3.12) which can be installed in parallel +* Platform-python == Python3 + - Required for dnf +* Python provided as separate packages (python3.11, python3.12) + - Installable in parallel --- @@ -232,42 +301,40 @@ Well... it depends ![](rfc1925-5.png) --- -## really quick backstory, I promise +## Really Quick Backstory, I Promise -* before: take great individual care of the servers in our backyard garden -* now: fully industrialized farm with machines to prepare, plant, harvest, etc - * yes this is cattle vs pets -* development cycles on the order of days, not months or quarters -* an EL release is supported for 10 years... but there's a new JS framework yesterday (probably) -* python, postgres, nodejs, nginx, php, ruby... some more swift than others -* containers? - - "It is easier to move a problem around than it is to solve it." - - can just use any other container distro -* what if the user could compose their "own" OS? +* Before: Carefully managed servers like a backyard garden +* Now: Fully industrialized farm with automated preparation, planting, and harvesting +* Cattle vs. Pets analogy +* Development cycles now measured in days +* EL releases need to be supported for 10 years, but new tech evolves rapidly (e.g., new JS frameworks) +* Diverse tech stack: Python, Postgres, Node.js, Nginx, PHP, Ruby (some evolving faster than others) +* "It is easier to move a problem around than it is to solve it." +* What if users could compose their "own" OS? --- @@ -277,15 +344,25 @@ turns out that becomes an multiplicatively awful problem. almost no modules were --- -## so we're doomed? +## So We're Doomed?
@@ -294,11 +371,11 @@ Imagine the most cyclical digraph you can think of. Now make it ten times worse.
-* can't package everything -* **can** maintain the important stuff -* software velocity isn't going to decrease -* we humans can only get so fast -* focus on tooling +* Can't package everything +* **Can** maintain the important stuff +* Software velocity isn't slowing down +* We humans are only so fast +* Focus on improving tooling
@@ -314,39 +391,115 @@ Imagine the most cyclical digraph you can think of. Now make it ten times worse. --- +Even though some of these tools have undergone updates or replacements, they remain fundamental to our processes. -* empanadas... koji/mock/mbs -> distrobuild (python) -> peridot (go, python, rust) -* its us, we're the problem -* "how do we serve it to users? build images?" i.e. release engineering -* toolkit began as a bunch of scripts - * began compiling into empanadas - * Empanadas is a X that Y... @TODO: figure this out :) -* pv2 - importer -* apollo -* mirrormanager -* mailman +To gain better control over our release builds, we created distrobuild as a wrapper around koji and MBS. + +We've also been using PV2 for automating upstream imports during our transition from Go to Python with our build system (Peridot). + +This is a big WIP still, but there's good things on the horizon. + +Additionally, Apollo is our errata feed suite, providing UI and workflows to fetch updates. However, it still requires significant improvements. +--> + +## Building Rocky Linux +* Leveraging Fedora tools: koji, mock, MBS, pungi, lorax +* Distrobuild: enhances koji/MBS control +* PV2: used for automating upstream imports +* Recognizing our challenges: "It's us, we're the problem" +* Apollo: errata publisher requiring improvements --- - -# summary ? +## empanadas (git.resf.org/sig_core/toolkit) +
+ +
+ +
+ +* empandas is **the** way Rocky does release engineering and image building +* toolkit began as a bunch of (bash) scripts + * began compiling into empanadas as a python CLI +* click, rpm python, ... `subprocess.run()` + * like an embarrassing amount of `subprocess.run()` +* I don't claim to be a good Python dev +* some refactoring efforts going on + +
+ --- -# Q & A + + +## Friends + +* mirrormanager +* mailman/hyperkitty +* kiwi +* others I've surely forgotten + +--- + + + +# What does it all mean? + +* Python is core to Fedora, CentOS, and Rocky Linux distributions +* Tremendous amount of Python used in building and distributing the OS +* Python is approachable for "sysadmin-adjacent" roles, even if the code isn’t always Pythonic +* RFC-1925 always has an answer to any question --- # Thank You! +--- +# Q & A +--- + +
+ +
+ +## Join us on Mattermost! https://chat.rockylinux.org + +
+ + + +
+ +
- neil@shrug.pw / neil@resf.org - @kneel.bsky.social - [thepotato.tech](https://thepotato.tech) +
+ ---