sleep helps
This commit is contained in:
Neil Hanlon
parent
a8c0b899d6
commit
7859238d09
GPG Key ID:
705BC21EC3C70F34
Binary file not shown.
|
After Width: | Height: | Size: 57 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 47 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 177 KiB |
@ -8,23 +8,26 @@ footer: Rocky Enterprise Software Foundation
|
||||
---
|
||||
|
||||
# Python and Rocky Linux
|
||||
## (a love story in three parts)
|
||||
<div data-marpit-fragment>
|
||||
<h4>(a love story in three parts)</h4>
|
||||
</div>
|
||||
|
||||
---
|
||||
<!-- paginate: true -->
|
||||
<!-- footer: "" -->
|
||||
<!-- header: "" -->
|
||||
## Who am I?
|
||||
|
||||
<!--
|
||||
My name is Neil Hanlon. I'm one of the founders of Rocky Linux and the infra team lead.
|
||||
|
||||
I work for CIQ in the Open Source Program Office where I primarily focus on the Rocky Linux community and infrastructure to make it the best it can be.
|
||||
|
||||
I'm also a Fedora packager, contributor to OpenStack-Ansible, and in a former life, I used to be a network engineer.
|
||||
|
||||
My name is Neil Hanlon, and I'm one of the founders of Rocky Linux, serving as the infrastructure team lead.
|
||||
I work at CIQ in the Open Source Program Office, where I primarily focus on the Rocky Linux community and infrastructure, striving to make it the best it can be.
|
||||
I'm also a Fedora packager, a contributor to OpenStack-Ansible, and in a former life, I was a network engineer.
|
||||
-->
|
||||
|
||||
|
||||
|
||||
<div data-marpit-fragment>
|
||||
|
||||
### "Professionally"
|
||||
|
||||
- Rocky Linux cofounder & infra lead
|
||||
@ -33,12 +36,21 @@ I'm also a Fedora packager, contributor to OpenStack-Ansible, and in a former li
|
||||
- OpenStack-Ansible reviewer/contributor
|
||||
- Menace to society (IPv6 Zealot)
|
||||
|
||||
</div>
|
||||
|
||||
<div data-marpit-fragment>
|
||||
|
||||
### "Unprofessionally"
|
||||
|
||||
- Plays guitar and trumpet
|
||||
- Tinkerer, HAM (KC1UYE), pretend electrician
|
||||
- Tinkerer, HAM (KC1UYE)
|
||||
- pretend electrician
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
<!-- header: Python and Rocky Linux -->
|
||||
<!-- footer: Rocky Enterprise Software Foundation -->
|
||||
# What is Rocky Linux?
|
||||
|
||||
<!--
|
||||
@ -48,49 +60,65 @@ So.. you may be asking yourself.. What the heck _is_ Rocky Linux?
|
||||
---
|
||||
|
||||
<!--
|
||||
Rocky Linux was founded in December 2020 in response to the CentOS project's shift in focus from CentOS Linux to CentOS Stream, which no longer rebuilds RHEL but tracks ahead of its next release.
|
||||
|
||||
Rocky Linux was founded in December of 2020 as a result of the shift in focus for the CentOS project from CentOS Linux to CentOS Stream--no longer rebuilding RHEL, but rather tracking ahead of the next release of it.
|
||||
|
||||
RHEL is used widely in the industry for misison-critical enterprise services--from x86_64 and aarch64 to powerpc and s390x mainframes.
|
||||
|
||||
Rocky exists to fill the gap left by CentOS Linux's absence, as a free rebuild of RHEL that aims to be entirely compatible with the upstream distribution. Some of those gaps have python shaped holes.
|
||||
RHEL is widely used in the industry for mission-critical enterprise services, supporting architectures from x86_64 and aarch64 to powerpc and s390x mainframes.
|
||||
|
||||
Rocky Linux exists to fill the gap left by CentOS Linux's absence, offering a free rebuild of RHEL that aims to be entirely compatible with the upstream distribution. Some of those gaps have Python-shaped holes.
|
||||
-->
|
||||
|
||||
# Rocky Linux is a community-driven Enterprise Linux distribution--stable enough for the largest enterprise to rely on it, and community-driven to ensureit stays accessible to all
|
||||
|
||||
---
|
||||
<!-- dfooter: (ref: https://x.com/carlwgeorge/status/1439724296742576130/) -->
|
||||
<!-- footer: "" -->
|
||||
<!-- header: "" -->
|
||||
|
||||
<!--
|
||||
The Rocky Enterprise Software Foundation is the legal entity and fiscal host for Rocky Linux. It's purpose is to ensure the longevity of Rocky Linux according to our founding principals of transparency.
|
||||
|
||||
@TODO: add in details about what Rocky and its lifecycle is. talk about 8 / 9 , stream, and 10.
|
||||
Let's discuss what Rocky Linux is and its lifecycle, particularly in the context of Enterprise Linux.
|
||||
After the abrupt end-of-life for CentOS Linux 8 and the shift in focus to CentOS Stream, Rocky Linux emerged to fill the gap.
|
||||
Rocky is built from a minimized subset of the Fedora package set.
|
||||
A new major release occurs every three years, which corresponds to roughly six Fedora releases.
|
||||
Each major release is supported for 10 years; the first half with full support and the latter half with 'Maintenance' support.
|
||||
We also leverage community-supported addon repositories like EPEL and rpmfusion to include many Python packages that are not in the base OS.
|
||||
-->
|
||||
|
||||
# The vision of the Rocky Enterprise Software Foundation is to create and nurture a community of individuals and organizations that are committed to ensuring the longevity, stewardship, and innovation of enterprise-class open-source software
|
||||
## Enterprise Linux?
|
||||
|
||||
<div data-marpit-fragment>
|
||||
<img src="E_rTivEXMAg6Tjh.png" style="display: block; width: 76%; margin: 0 auto 0.5em auto;" />
|
||||
</div>
|
||||
|
||||
* CentOS Linux 8 EOL-ed early; focus shifted to CentOS Stream
|
||||
* Built from minimal subset of Fedora package set
|
||||
* New major release every 3 years (about six Fedora releases)
|
||||
* Each major release supported for 10 years
|
||||
- First half: Full support
|
||||
- Second half: 'Maintenance' support
|
||||
* Community-supported addon repositories like EPEL and rpmfusion
|
||||
- Many Python packages not included in the base OS
|
||||
|
||||
---
|
||||
<!-- header: Python and Rocky Linux -->
|
||||
<!-- footer: Rocky Enterprise Software Foundation -->
|
||||
|
||||
# Cool.. So what about the python stuff?
|
||||
|
||||
<!--
|
||||
I'm getting there! Good stories require exposition.
|
||||
-->
|
||||
<marquee direction=right><h1>:snake:</h1></marquee>
|
||||
|
||||
---
|
||||
|
||||
# Part I - The Operating System
|
||||
|
||||
<!--
|
||||
@TODO: fill in notes for self
|
||||
Let's start with Part I: The Operating System.
|
||||
In any modern Linux distribution, you'll find software written in a myriad of languages such as Java, Rust, C, C++, Ruby, Node.js, and more. However, core components of Fedora-based distributions, including Rocky Linux, often rely heavily on Python.
|
||||
While the list isn't exhaustive, I've selected a few key components that illustrate just how important Python is to the core of our system.
|
||||
-->
|
||||
|
||||
(in no particular order)
|
||||
|
||||
<div data-marpit-fragment>
|
||||
|
||||
* platform-python
|
||||
* dnf
|
||||
* dnf / rpm
|
||||
* anaconda (no, not that anaconda)
|
||||
* cockpit-project
|
||||
|
||||
@ -98,132 +126,173 @@ I'm getting there! Good stories require exposition.
|
||||
|
||||
---
|
||||
|
||||
## anaconda (pyanaconda)
|
||||
<!--
|
||||
Let's discuss Cockpit, a web-based graphical interface designed for managing servers.
|
||||
Cockpit uses a mix of Python and JavaScript to provide a user-friendly experience.
|
||||
It's available for a variety of distributions, including Debian and Fedora.
|
||||
Think of it as a modern, more user-friendly alternative to Webmin.
|
||||
Cockpit allows you to manage various server aspects such as networking, storage, logs, containers, VMs, and more.
|
||||
-->
|
||||
## Cockpit
|
||||
* Web-based graphical interface for servers
|
||||
* Built with Python and JavaScript
|
||||
* Available for Debian and Fedora
|
||||
* Think "Webmin but less bad"
|
||||
* Manages networking, storage, logs, containers, VMs, etc.
|
||||
|
||||
<!-- @TODO add image -->
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
@TODO: fill in notes for self
|
||||
Next, let's talk about Anaconda, also known as pyanaconda.
|
||||
Anaconda is the installation program used by Fedora, Red Hat, and others.
|
||||
It allows for both GUI (via VNC) and TUI-based interactions during the installation process.
|
||||
Additionally, Anaconda can process kickstart files for automated, unattended installations.
|
||||
It uses Gtk.Builder and Glade for customization and modifications of the UI elements.
|
||||
The UI is currently undergoing rewrites to smooth out some rough edges, likely to be seen in Fedora 42.
|
||||
Anaconda leverages the blivet library for disk setup and supports various addons like ostree and openscap.
|
||||
-->
|
||||
|
||||
* installation program for Fedora, Red Hat, others
|
||||
## Anaconda (pyanaconda)
|
||||
* Installation program for Fedora, Red Hat, others
|
||||
* GUI (VNC) or TUI based interaction
|
||||
* can also process kickstart file for unattended installations
|
||||
* Gtk.Builder / Glade - customization and modification
|
||||
* I actively avoid this because it's painful
|
||||
* UI parts being rewritten to have fewer rough edges
|
||||
- Fedora 42, probably
|
||||
* blivet - storage library used by anaconda for setting up server's disks
|
||||
* addons: ostree, openscap, etc
|
||||
* Supports kickstart files for unattended installations
|
||||
* Uses Gtk.Builder / Glade for customization
|
||||
* UI being rewritten for fewer rough edges
|
||||
- Expected in Fedora 42
|
||||
* Blivet: Storage library for disk setup
|
||||
* Addons: ostree, openscap, etc.
|
||||
|
||||
<!-- @TODO add image -->
|
||||
|
||||
---
|
||||
|
||||
## cockpit
|
||||
|
||||
<!--
|
||||
@TODO: fill in notes for self
|
||||
Now let's break down DNF, short for "Dandified Yum."
|
||||
It serves as the package manager for distributions like Fedora, RHEL, SUSE, and a few others.
|
||||
DNF is essentially a rewrite of `yum`, which itself was a rewrite of `yup`, and so on.
|
||||
It's a high-level tool that sits on top of `rpm` to manage packages.
|
||||
Introduced in 2013, dnf4 used a combination of Python and C/C++.
|
||||
For context, dnf5, which came out in 2018, is written in C++ and is currently used only in Fedora.
|
||||
We’ll focus on the more widely-used dnf4 for now.
|
||||
-->
|
||||
|
||||
* web-based graphical interface for servers
|
||||
* python, javascript
|
||||
* available for Debian as well as Fedora
|
||||
* think "webmin but less bad"
|
||||
* networking, storage, logs, containers/VMs, and so on
|
||||
## DNF ("Dandified Yum")
|
||||
* Package manager for Fedora, RHEL, SUSE, and others
|
||||
* Rewritten version of `yum`, which was a rewrite of `yup`
|
||||
* Layer on top of `rpm` for package management
|
||||
* DNF4 (2013): Python + C/C++
|
||||
- We don't talk about DNF 1-3...
|
||||
* DNF5 (2018): C++
|
||||
- Currently only in Fedora
|
||||
|
||||
---
|
||||
|
||||
## dnf ("dandified yum")
|
||||
|
||||
<!--
|
||||
@TODO: fill in notes for self
|
||||
Now, let's discuss DNF4 and its relationship with Python.
|
||||
There is quite a bit of history here, much of which can be quite dry.
|
||||
One significant change was moving to an external dependency solver, making the process faster and less memory-intensive.
|
||||
The transition from Python 2 to 3 was particularly challenging during this period.
|
||||
Several libraries play a role in DNF's functionality:
|
||||
- libdnf: Provides a high-level API for DNF and is written in C/C++.
|
||||
- libsolv: A package satisfiability solver also written in C.
|
||||
- librepo: A libcURL wrapper used for fetching Linux repository metadata and packages, with Python bindings available.
|
||||
- libcomps: An alternative to the slower python yum.comps library, also with Python bindings.
|
||||
-->
|
||||
|
||||
* package manager for Fedora, RHEL, SUSE, some others
|
||||
* rewrite of `yum` which is a rewrite of `yup`... etc
|
||||
* layer on top of `rpm` for package management
|
||||
* dnf4 (2013) - python + C/C++
|
||||
- we don't talk about dnf 1-3...
|
||||
* dnf5 (2018) - C++
|
||||
- only in Fedora right now
|
||||
### DNF4 and Python
|
||||
* Lots of probably boring history
|
||||
* External dependency solver: faster, less memory usage
|
||||
* Python 2/3 transition was (and is) challenging
|
||||
* libdnf: High-level API for DNF (C/C++)
|
||||
* libsolv: Package satisfiability solver (C)
|
||||
* librepo: libcURL wrapper for fetching repo metadata and packages (C/Python bindings)
|
||||
* libcomps: Alternative to slower python yum.comps library (C/Python bindings)
|
||||
|
||||
---
|
||||
<!-- header: "" -->
|
||||
<!-- footer: "" -->
|
||||
|
||||
### dnf4 and python
|
||||
## Platform-Python
|
||||
|
||||
<!--
|
||||
@TODO: fill in notes for self
|
||||
Let's talk about platform-python, a critical component for the stability of any EL-based distribution, including Rocky Linux.
|
||||
Every system needs a stable version of Python to rely on.
|
||||
For Rocky 8, this version is Python 3.6.8, and for Rocky 9, it's Python 3.9.18.
|
||||
Both are accessible via `/usr/libexec/platform-python`.
|
||||
Next, we'll look at some common pitfalls when using Python on EL systems.
|
||||
-->
|
||||
|
||||
* lots of probably boring history
|
||||
* external depsolver
|
||||
* python 2/3 transition period was (is) a dark time
|
||||
* libdnf
|
||||
- high level API for DNF (C/C++)
|
||||
* libsolv
|
||||
- package satisfiability solver (C)
|
||||
* librepo
|
||||
- libcURL wrapper for fetching Linux repo metadata and packages (C/Python bindings)
|
||||
* libcomps
|
||||
- alternative for slower python yum.comps library (C/Python bindings)
|
||||
|
||||
---
|
||||
|
||||
## platform-python
|
||||
|
||||
<!--
|
||||
@TODO: fill in notes for self
|
||||
-->
|
||||
|
||||
* system must have a stable python
|
||||
* rocky 8: 3.6.8
|
||||
* rocky 9: 3.9.18
|
||||
* /usr/libexec/platform-python
|
||||
* System must have a stable Python version
|
||||
* Rocky 8: Python 3.6.8
|
||||
* Rocky 9: Python 3.9.18
|
||||
* System python always located at `/usr/libexec/platform-python`
|
||||
|
||||
<div data-marpit-fragment>
|
||||
|
||||
## python on EL - pitfalls
|
||||
|
||||
* tools like ansible are easily confused
|
||||
- may need to force ansible_interpreter
|
||||
* many python packages/modules are built against specific pythons
|
||||
* RPM modularity makes this problem worse
|
||||
* most people just end up installing things using pip :(
|
||||
* we don't really have a good solution for this :/
|
||||
## Python on EL - Pitfalls
|
||||
* Tools like Ansible may get confused
|
||||
- May need to force `ansible_interpreter`
|
||||
* Many Python packages/modules built against specific Python versions
|
||||
* RPM modularity exacerbates the issue
|
||||
* Many end up using pip for package installation :(
|
||||
* No solid solution yet
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
<!-- header: Python and Rocky Linux -->
|
||||
<!-- footer: Rocky Enterprise Software Foundation -->
|
||||
|
||||
## using / developing with python on Rocky
|
||||
## Using / Developing with Python on Rocky
|
||||
|
||||
<!--
|
||||
Let's now discuss the experience of using and developing with Python on Rocky Linux.
|
||||
|
||||
It would be logical to ask, then.. how hard is it to use and develop using Python on Rocky Linux?
|
||||
How hard is it really to get started and be productive with Python on our platform?
|
||||
|
||||
Well... it depends
|
||||
The short answer is, like most things: it depends.
|
||||
|
||||
@TODO: finish speaker notes
|
||||
While there are hurdles, such as non-standard repositories and the intricacies of building RPM packages, tools like pkgs.org can help find what you need. For many, pip remains a go-to resource for managing Python packages.
|
||||
|
||||
-->
|
||||
|
||||
* it depends!
|
||||
* harder than it could be. way more frustrating than it should be
|
||||
* pkgs.org can help find things in non-standard repos (EPEL, rpmfusion)
|
||||
* building RPM packages isn't that difficult, but pip is probably your best friend
|
||||
<div data-marpit-fragment>
|
||||
|
||||
### How Hard Is It?
|
||||
* It depends!
|
||||
* More challenging and frustrating than ideal
|
||||
* Use pkgs.org for finding non-standard repos (EPEL, rpmfusion)
|
||||
* Building RPMs isn't too hard, but pip is often your best friend
|
||||
|
||||
</div>
|
||||
|
||||
<!-- @TODO add image -->
|
||||
|
||||
---
|
||||
|
||||
### python on Rocky
|
||||
### Python on Rocky
|
||||
|
||||
<!--
|
||||
Next, let's take a closer look at how Python is handled in Rocky 8 and Rocky 9.
|
||||
|
||||
Understanding the differences between these versions can help you navigate the complexities of developing with Python on Rocky Linux.
|
||||
|
||||
In Rocky 8, you have both modular and non-modular Python versions. Platform-python is non-modular and is crucial for dnf.
|
||||
|
||||
However, most Python packages and modules are built against the default Python version.
|
||||
|
||||
In Rocky 9, platform-python is synonymous with Python3 and is required for dnf. Instead of modules, Python is now provided as separately named packages, like python3.11 and python3.12, which can be installed in parallel.
|
||||
-->
|
||||
|
||||
#### Rocky 8
|
||||
|
||||
* modular **and** non-modular python
|
||||
* platform-python for dnf is non-modular
|
||||
* most python packages/modules only built against default python
|
||||
|
||||
* Modular and non-modular Python versions
|
||||
* Platform-python (non-modular) required for dnf
|
||||
* Most packages/modules built against default Python
|
||||
* Modular python not modularized "correctly"
|
||||
#### Rocky 9
|
||||
|
||||
* platform-python == python3
|
||||
- required by dnf
|
||||
* python not provided as modules but as seperately named packages (python3.11, python3.12) which can be installed in parallel
|
||||
* Platform-python == Python3
|
||||
- Required for dnf
|
||||
* Python provided as separate packages (python3.11, python3.12)
|
||||
- Installable in parallel
|
||||
|
||||
---
|
||||
<!-- footer: (ref: https://www.rfc-editor.org/rfc/rfc1925)-->
|
||||
@ -232,42 +301,40 @@ Well... it depends
|
||||
|
||||
|
||||
<!--
|
||||
@TODO: fill in notes for self
|
||||
Up next, we'll dive into RPM modularity. This concept was introduced to address the growing complexity and diversity of software requirements in modern enterprise environments.
|
||||
|
||||
It aims to provide a way to customize and manage different versions of software components within the same distribution.
|
||||
|
||||
By allowing users to select and combine different modules, we hoped to offer greater flexibility and control over their systems.
|
||||
|
||||
However, as we will discuss, implementing RPM modularity brought its own set of challenges.
|
||||
-->
|
||||
|
||||
---
|
||||
|
||||
## really quick backstory, I promise
|
||||
## Really Quick Backstory, I Promise
|
||||
|
||||
<!--
|
||||
In the beginning, our installation base was like managing a small garden with a few servers, carefully maintained within a controlled environment. Development cycles were long, and software ran on system-installed packages.
|
||||
|
||||
@TODO: revise these notes for brevity and clarity
|
||||
Today, it's like managing an industrialized farm. The sheer scale makes it impossible to manage servers individually. Development cycles are now measured in days, not months or quarters, with parallel systems having disjoint dependencies.
|
||||
|
||||
In the beginning we had servers in our gardens. A handful in our walled off space that we controlled. We had long development cycles and software which just ran using the packages on the system.
|
||||
For an enterprise distribution promising 10 years of support, this creates a vicious cycle: fewer, outdated packages and missing dependencies lead to a bad developer experience, fewer developers, fewer users, and less funding.
|
||||
|
||||
Fast forward to today, and we're on a fully industrialized farm where it's impracticable to treat our systems like they're in a garden. The scope and scale simply make it a losing battle. Now, development cycles are on the order of days, not months; and we're deploying parallel systems with entirely disjoint dependencies as a matter of best practices.
|
||||
Containers might seem like a solution but often just shift the problem without solving it. They make the enterprise distribution merely a base for running different containerized environments.
|
||||
|
||||
For an enterprise distribution that supports a snapshot of software for 10 years, then--this leads vicious cycle of fewer, outdated packages and missing dependencies, leading to bad devex, fewer devs, fewer users, less money, GOTO 1.
|
||||
The idea of modularity—allowing users to customize their OS components—emerged but proved problematic. Modules were rarely isolated, needed individual bootstrapping per stream, and required extensive testing.
|
||||
|
||||
Containers save the day?
|
||||
|
||||
- not really... this just makes the 'Enterprise' distribution just a way to run a container that isn't the enterprise distr
|
||||
|
||||
so.. modularity became an idea: "What if the customer could just customize the OS components they need?"
|
||||
|
||||
turns out that becomes an multiplicatively awful problem. almost no modules were isolated, require individual bootstrapping per stream, have to test with all modules, etc
|
||||
-->
|
||||
|
||||
* before: take great individual care of the servers in our backyard garden
|
||||
* now: fully industrialized farm with machines to prepare, plant, harvest, etc
|
||||
* yes this is cattle vs pets
|
||||
* development cycles on the order of days, not months or quarters
|
||||
* an EL release is supported for 10 years... but there's a new JS framework yesterday (probably)
|
||||
* python, postgres, nodejs, nginx, php, ruby... some more swift than others
|
||||
* containers?
|
||||
- "It is easier to move a problem around than it is to solve it."
|
||||
- can just use any other container distro
|
||||
* what if the user could compose their "own" OS?
|
||||
* Before: Carefully managed servers like a backyard garden
|
||||
* Now: Fully industrialized farm with automated preparation, planting, and harvesting
|
||||
* Cattle vs. Pets analogy
|
||||
* Development cycles now measured in days
|
||||
* EL releases need to be supported for 10 years, but new tech evolves rapidly (e.g., new JS frameworks)
|
||||
* Diverse tech stack: Python, Postgres, Node.js, Nginx, PHP, Ruby (some evolving faster than others)
|
||||
* "It is easier to move a problem around than it is to solve it."
|
||||
* What if users could compose their "own" OS?
|
||||
|
||||
---
|
||||
<!-- footer: (source: https://youtu.be/F5SWz3yPXjo ; The Self Abolition of Enterprise Linux Distributions; Dan Čermák, SUSE) -->
|
||||
@ -277,15 +344,25 @@ turns out that becomes an multiplicatively awful problem. almost no modules were
|
||||
<!--
|
||||
Imagine the most cyclical digraph you can think of. Now make it ten times worse.
|
||||
|
||||
@TODO: fill in notes for self
|
||||
This slide illustrates the seemingly endless loop of problems and solutions that come with maintaining an enterprise Linux distribution.
|
||||
|
||||
We have to juggle software velocity, modularity complexities, and the ever-evolving landscape of dependencies and packages.
|
||||
|
||||
As we discussed earlier with RPM modularity, this can lead to a multiplicative number of challenges that make maintenance highly complex.
|
||||
|
||||
Let’s dig deeper into how these issues create a vicious cycle and what possible strategies we can employ to mitigate these challenges.
|
||||
-->
|
||||
|
||||
---
|
||||
|
||||
## so we're doomed?
|
||||
## So We're Doomed?
|
||||
|
||||
<!--
|
||||
@TODO: fill in notes for self
|
||||
So, are we doomed?
|
||||
We face challenges in packaging and maintaining all the software needed for a modern distro, but we can focus on maintaining the most essential components.
|
||||
Software development velocity isn't going to slow down, and as humans, we can only get so fast.
|
||||
Our best strategy is to focus on improving our tooling to enhance efficiency and automation.
|
||||
e.g. here's a diagram from dan cermak from suse discussing how we could have an tight, iterative pypi-RPM build loop
|
||||
-->
|
||||
|
||||
<div data-marpit-fragment style="display: inline-flex; width: 50%; align-self: end">
|
||||
@ -294,11 +371,11 @@ Imagine the most cyclical digraph you can think of. Now make it ten times worse.
|
||||
|
||||
<div style="display: inline-flex; width: 50%; align-self: start; margin-top: -8em;">
|
||||
|
||||
* can't package everything
|
||||
* **can** maintain the important stuff
|
||||
* software velocity isn't going to decrease
|
||||
* we humans can only get so fast
|
||||
* focus on tooling
|
||||
* Can't package everything
|
||||
* **Can** maintain the important stuff
|
||||
* Software velocity isn't slowing down
|
||||
* We humans are only so fast
|
||||
* Focus on improving tooling
|
||||
|
||||
</div>
|
||||
|
||||
@ -314,39 +391,115 @@ Imagine the most cyclical digraph you can think of. Now make it ten times worse.
|
||||
---
|
||||
|
||||
<!--
|
||||
The tools that we use to build Rocky Linux are heavily influenced by Fedora's tools, many of which are written in Python due to its approachability and versatility.
|
||||
|
||||
@TODO: finish notes, expand this into more than a slide for clarity
|
||||
Initially, we adopted various Fedora tools such as koji, mock, MBS, pungi, and lorax to compose the OS.
|
||||
- Pungi serves as the dependency solver and compose maker.
|
||||
- Lorax helps in creating images (ISOs) and can perform other custom tasks.
|
||||
|
||||
Even though some of these tools have undergone updates or replacements, they remain fundamental to our processes.
|
||||
|
||||
To gain better control over our release builds, we created distrobuild as a wrapper around koji and MBS.
|
||||
|
||||
We've also been using PV2 for automating upstream imports during our transition from Go to Python with our build system (Peridot).
|
||||
|
||||
This is a big WIP still, but there's good things on the horizon.
|
||||
|
||||
Additionally, Apollo is our errata feed suite, providing UI and workflows to fetch updates. However, it still requires significant improvements.
|
||||
-->
|
||||
|
||||
* empanadas... koji/mock/mbs -> distrobuild (python) -> peridot (go, python, rust)
|
||||
* its us, we're the problem
|
||||
* "how do we serve it to users? build images?" i.e. release engineering
|
||||
* toolkit began as a bunch of scripts
|
||||
* began compiling into empanadas
|
||||
* Empanadas is a X that Y... @TODO: figure this out :)
|
||||
* pv2 - importer
|
||||
* apollo
|
||||
* mirrormanager
|
||||
* mailman
|
||||
## Building Rocky Linux
|
||||
* Leveraging Fedora tools: koji, mock, MBS, pungi, lorax
|
||||
* Distrobuild: enhances koji/MBS control
|
||||
* PV2: used for automating upstream imports
|
||||
* Recognizing our challenges: "It's us, we're the problem"
|
||||
* Apollo: errata publisher requiring improvements
|
||||
|
||||
---
|
||||
|
||||
# summary ?
|
||||
## empanadas (git.resf.org/sig_core/toolkit)
|
||||
|
||||
<!--
|
||||
@TODO: Wrap up
|
||||
Empandas is central to how we handle release engineering and image building in Rocky Linux.
|
||||
Originally, our toolkit started as a series of bash scripts.
|
||||
Over time, we consolidated these scripts into Empandas, a Python CLI.
|
||||
The CLI relies on packages like click and rpm python, with a substantial amount of `subprocess.run()` calls for executing shell commands.
|
||||
There's an excessive and somewhat embarrassing reliance on `subprocess.run()`.
|
||||
I don't consider myself a particularly skilled Python developer, and there are ongoing refactoring efforts to improve the codebase.
|
||||
-->
|
||||
|
||||
<div style="display: inline-flex; width: 300px; align-self: end; margin-top: 6em;">
|
||||
<img src=empanadas.png width=300px align=right />
|
||||
</div>
|
||||
|
||||
<div style="display: inline-flex; width: 100%; align-self: start; margin-top: -14em;">
|
||||
|
||||
* empandas is **the** way Rocky does release engineering and image building
|
||||
* toolkit began as a bunch of (bash) scripts
|
||||
* began compiling into empanadas as a python CLI
|
||||
* click, rpm python, ... `subprocess.run()`
|
||||
* like an embarrassing amount of `subprocess.run()`
|
||||
* I don't claim to be a good Python dev
|
||||
* some refactoring efforts going on
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
# Q & A
|
||||
<!--
|
||||
Several key tools and services are integral to our operations in Rocky Linux.
|
||||
Mirrormanager helps us manage our network of mirror servers for efficient distribution.
|
||||
Mailman and Hyperkitty are essential for our mailing list and discussion management.
|
||||
We also use KIWI for creating and managing appliance images.
|
||||
There are other important tools and services that contribute to our processes, though I might not recall all of them off the top of my head.
|
||||
-->
|
||||
|
||||
## Friends
|
||||
|
||||
* mirrormanager
|
||||
* mailman/hyperkitty
|
||||
* kiwi
|
||||
* others I've surely forgotten
|
||||
|
||||
---
|
||||
|
||||
<!--
|
||||
What does it all mean?
|
||||
Python plays a fundamental role not only in Fedora, CentOS, and Rocky Linux distributions but also in the processes involved in building and distributing these OSes.
|
||||
Python's approachability makes it well-known among "sysadmin-adjacent" individuals, even if they don't always write Pythonic code.
|
||||
Lastly, we can always refer to RFC-1925 for answers to any challenging questions even when they're not related to networking.
|
||||
-->
|
||||
|
||||
# What does it all mean?
|
||||
|
||||
* Python is core to Fedora, CentOS, and Rocky Linux distributions
|
||||
* Tremendous amount of Python used in building and distributing the OS
|
||||
* Python is approachable for "sysadmin-adjacent" roles, even if the code isn’t always Pythonic
|
||||
* RFC-1925 always has an answer to any question
|
||||
|
||||
---
|
||||
# Thank You!
|
||||
---
|
||||
# Q & A
|
||||
---
|
||||
|
||||
<div style="display: inline-flex; align-self: end; margin-top: 2em;">
|
||||
|
||||
<div style="width: 60%">
|
||||
|
||||
## Join us on Mattermost! https://chat.rockylinux.org
|
||||
|
||||
</div>
|
||||
|
||||
<img src=chat.rockylinux.org.png width=200px align=right style="margin-left: -2em; display: block;" />
|
||||
|
||||
</div>
|
||||
|
||||
<div style="display: inline-flex; width: 100%; align-self: start; margin-top: -14em;">
|
||||
|
||||
- neil@shrug.pw / neil@resf.org
|
||||
- @kneel.bsky.social
|
||||
- [thepotato.tech](https://thepotato.tech)
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
Loading…
Reference in New Issue
Block a user