danigm.net - linux

Python 3.13 Beta 1

danigm — Wed, 22 May 2024 00:00:00 +0200

Python 3.13 beta 1 is out, and I've been working on the openSUSE Tumbleweed package to get it ready for the release.

Installing python 3.13 beta 1 in Tumbleweed

If you are adventurous enough to want to test the python 3.13 and you are using openSUSE Tumbleweed, you can give it a try and install the current devel package:

# zypper addrepo -p 1000 https://download.opensuse.org/repositories/devel:languages:python:Factory/openSUSE_Tumbleweed/devel:languages:python:Factory.repo
# zypper refresh
# zypper install python313

What's new in Python 3.13

Python interpreter is pretty stable nowadays and it doesn't change too much to keep code compatible between versions, so if you are writing modern Python, your code should continue working whit this new version. But it's actively developed and new versions have cool new functionalities.

New and improved interactive interpreter, colorized prompts, multiline editing with history preservation, interactive help with F1, history browsing with F2, paste mode with F3.
A set of performance improvements.
Removal of many deprecated modules: aifc, audioop, chunk, cgi, cgitb, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, xdrlib, lib2to3.

Enabling Experimental JIT Compiler

The python 3.13 version will arrive with an experimental functionality to improve performance. We're building with the --enable-experimental-jit=yes-off so it's disabled by default but it can be enabled with a virtualenv before launching:

$ PYTHON_JIT=1 python3.13

Free-threaded CPython

The python 3.13 has another build option to disable the Global Interpreter Lock (--disable-gil), but we're not enabling it because in this case it's not possible to keep the same behavior. Building with disabled-gil will break compatibility.

In any case, maybe it's interesting to be able to provide another version of the interpreter with the GIL disabled, for specific cases where the performance is something critical, but that's something to evaluate.

We can think about having a python313-nogil package, but it's not something trivial to be able to have python313 and python313-nogil at the same time in the same system installation, so I'm not planning to work on that for now.

Where's my python code?

danigm — Sat, 03 Feb 2024 00:00:00 +0100

Python is a interpreted language, so the python code are just text files with the .py extension. For simple scripts it's really easy to have your files located, but when you starts to use dependencies and different projects with different requirements the thing starts to get more complex.

PYTHONPATH

The Python interpreter uses a list of paths to try to locate python modules, for example this is what you can get in a modern GNU/Linux distribution by default:

Python 3.11.7 (main, Dec 15 2023, 10:49:17) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['',
 '/usr/lib64/python311.zip',
 '/usr/lib64/python3.11',
 '/usr/lib64/python3.11/lib-dynload',
 '/usr/lib64/python3.11/site-packages',
 '/usr/lib64/python3.11/_import_failed',
 '/usr/lib/python3.11/site-packages']

These are the default paths where the python modules are installed. If you install any python module using your linux packaging tool, the python code will be placed inside the site-packages folder.

So system installed python modules can be located in:

/usr/lib/python3.11/site-packages for modules that are architecture independent (pure python, all .py files)
/usr/lib64/python3.11/site-packages for modules that depends on the arquitecture, that's something that uses low level libraries and needs to build so there are some .so files.

pip

When you need a new python dependency you can try to install from your GNU/Linux distribution using the default package manager like zypper, dnf or apt, and those python files will be placed in the system paths that you can see above.

But distributions doesn't pack all the python modules and even if they do, you can require an specific version that's different from the one packaged in your favourite distribution, so in python it's common to install dependencies from the Python Package Index (PyPI).

Python has a tool to install and manage Python packages that looks for desired python modules in PyPI.

You can install new dependencies with pip just like:

$ pip install django

And that command looks for the django python module in the PyPI, downloads and install it, in your user $HOME/.local/lib/python3.11/site-packages folder if you use --user, or in a global system path like /usr/local/lib or /usr/lib if you run pip as root.

But the usage of pip directly in the system is something not recommended today, and even it's disabled in some distributions, like openSUSE Tumbleweed.

[danigm@localhost ~] $ pip install django
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try
    zypper install python311-xyz, where xyz is the package
    you are trying to install.

    If you wish to install a non-rpm packaged Python package,
    create a virtual environment using python3.11 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.

    If you wish to install a non-rpm packaged Python application,
    it may be easiest to use `pipx install xyz`, which will manage a
    virtual environment for you. Install pipx via `zypper install python311-pipx` .

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

virtualenvs

Following the current recommendation, the correct way of installing third party python modules is to use virtualenvs.

The virtualenvs are just specific folders where you install your python modules and some scripts that make's easy to use it in combination with your system libraries so you don't need to modify the PYTHONPATH manually.

So if you've a custom project and want to install python modules you can create your own virtualenv and use pip to install dependencies there:

[danigm@localhost tmp] $ python3 -m venv myenv
[danigm@localhost tmp] $ . ./myenv/bin/activate
(myenv) [danigm@localhost tmp] $ pip install django
Collecting django
...
Successfully installed asgiref-3.7.2 django-5.0.1 sqlparse-0.4.4

So all dependencies are installed in my new virtualenv folder and if I use the python from the virtualenv it's using those paths, so all the modules installed there are usable inside that virtualenv:

(myenv) [danigm@localhost tmp] $ ls myenv/lib/python3.11/site-packages/django/
apps  contrib  db        forms  __init__.py  middleware   shortcuts.py  templatetags  urls   views
conf  core     dispatch  http   __main__.py  __pycache__  template      test          utils
(myenv) [danigm@localhost tmp] $ python3 -c "import django; print(django.__version__)"
5.0.1
(myenv) [danigm@localhost tmp] $ deactivate

With virtualenvs you can have multiple python projects, with different dependencies, isolated, so you use different dependencies when you activate your desired virtualenv:

activate $ . ./myenv/bin/activate
deactivate $ deactivate

High level tools to handle virtualenvs

The venv module is a default Python module and as you can see above, it's really simple to use, but there are some tools that provides some tooling around it, to make it easy for you, so usually you don't need to use venv directly.

pipx

For final python tools, that you are not going to use as dependencies in your python code, the recommended tool to use is pipx.

The tool creates virtualenv automatically and links the binaries so you don't need to worry about anything, just use as a way to install third party python applications and update/uninstall using it. The pipx won't mess your system libraries and each installation will use a different virtualenv, so even tools with incompatible dependencies will work nicely together in the same system.

Libraries, for Python developers

In the case of Python developers, when you need to manage dependencies for your project, there are a lot of nice high level tools for managing dependencies.

These tools provides different ways of managing dependencies, but all of them relies in the use of venv, creating the virtualenv in different locations and providing tools to enable/disable and manage dependencies inside those virtualenvs.

For example, poetry creates virtualenvs by default inside the .cache folder, in my case I can find all poetry created virtualenvs in:

/home/danigm/.cache/pypoetry/virtualenvs/

Most of these tools add other utilities on top of the dependency management. Just for installing python modules easily you can always use default venv and pip modules, but for more complex projects it's worth to investigate high level tools, because it'll make easy to manage your project dependencies and virtualenvs.

Conclusion

There are a lot of python code inside any modern Linux distribution and if you're a python developer it's possible to have a lot of python code. Make sure to know the source of your modules and do not mix different environments to avoid future headaches.

As a final trick, if you don't know where's the actual code of some python module in your running python script, you can always ask:

>>> import django
>>> django.__file__
'/tmp/myenv/lib64/python3.11/site-packages/django/__init__.py'

This could be even more complicated if you start to use containers and different python versions, so keep you dependencies clean and up to date and make sue that you know where is your Python code.

Updating GNOME shell extensions to GNOME 45

danigm — Fri, 03 Nov 2023 00:00:00 +0100

The new version of the GNOME desktop was released more than one month ago. It takes some time to arrive to the final user, because distributions should integrate, tests and release the new desktop, and that's not something simple, and it should integrate in the distribution release planning.

So right now, it's possible that there's just a few people with the latest version of the desktop right now, just people with rolling release distros or people using testing versions of mayor distributions.

This is one of the reasons because a lot of gnome-shell extensions aren't updated to work with the latest version of GNOME, even after a few months, because even developers doesn't have the latest version of the desktop and it's not something "easy" to install, without a virtual machine or something like that. Even if the update is just a change in the metadata.json, there should be someone to test the extension, and even someone to request this update, and that will happen once the mayor distributions release a new version.

I'm using Tumbleweed, that's a rolling release and GNOME 45 is here just after the official release, but of course, a lot of extensions are not working, and in the infamous list of non working extensions there where the three that I'm maintaining right now:

mute-spotify-ads, extension to mute the spotify app when it's playing ads.
calc, a simple calculator in the alt+F2 input.
hide-minimized, hide minimized windows from alt+tab and overview.

The correct way

The correct way to maintain and update a gnome-shell extension should be to test and update "before" the official release of GNOME. It should be something easy for me, using Tumbleweed I can just add the GNOME Next repository, install the new desktop when it's in beta stage, and update the extension there.

But that's something that require an active maintainership... And right now what I do is to update the extensions when they are broken for me, so just when I need them, and that's after I get the new desktop and I find some time to update.

I know that's not the correct way and this produces a bad experience for other people using the extensions, but this is the easier thing to do for me. Maybe in the future I can do it correctly, and provide a tested update before the official release, maybe using snapper to be able to go back to stable, without the need of using virtual machines.

Update your extension to GNOME 45

The update to GNOME 45 was a bit more complex than previous ones. This version of the shell and gjs change a lot of things, so the migration it's not just to add a new number to the metadata.json, but requires incompatible changes, so extensions that works for GNOME 45 won't work for previous versions and vice versa.

But the people working in gnome-shell does a great work documenting and there's a really nice guide about how to upgrade your extension.

The most important part is the import section, that now it has a different syntax.

// Before GNOME 45
const Main = imports.ui.main;

// GNOME 45
import * as Main from 'resource:///org/gnome/shell/ui/main.js';

And some changes in the extension class, that now can inherit from existing ones to provide common usage, like preferences window:

import Adw from 'gi://Adw';

import {ExtensionPreferences, gettext as _} from 'resource:///org/gnome/Shell/Extensions/js/extensions/prefs.js';

export default class MyExtensionPreferences extends ExtensionPreferences {
    fillPreferencesWindow(window) {
        window._settings = this.getSettings();

        const page = new Adw.PreferencesPage();

        const group = new Adw.PreferencesGroup({
            title: _('Group Title'),
        });
        page.add(group);

        window.add(page);
    }
}

One year of Tumbleweed

danigm — Fri, 06 Oct 2023 00:00:00 +0200

More than a year has passed since I switched to openSUSE Tumbleweed Linux distribution, in both, my work computer (for obvious reasons) and in my personal computer and I can say that I'm really happy with the change.

Tumbleweed is a rolling release distribution, and in this kind of distributions there are a lot of changes every week, if you want the latest software, this kind of distribution is the way to go. But with high update frequency you are exposed to some kind of instability, it's impossible to have the latest changes without some broken program here and there, because not everyone is able to follow upstream changes without some weeks or months to update.

My distro history

I've been always a Linux user, since I get my first computer at 2003. In those days I was using a debian base distribution called knoppix. Then I switched to ubuntu when it appeared around 2004. But at that time I was a computer science student and I was exploring the whole free software and Linux ecosystem, so I was changing my distribution every time that I found a new one.

Like a lot of distro-hoppers, at some point I landed at ArchLinux and there I discovered the rolling release concept. And that was my home for some time, it was nice to have the latest available software just after the release.

At some point I bough a new computer and it was too new to work correctly with the kernel distributed in ArchLinux, so I tried different distributions and at that moment Fedora was the distro that works without too much complications with that computer, so I picked that one.

In 2019 I started to work at Endless and at that time I should try the EndlessOS, so I played a bit with the dual boot, having Fedora and EndlessOS at the same time. That was the first time that I get in contact with immutable distributions, something that's getting more popular everyday, but this distributions rely a lot on containers (flatpak, podman) and, even being something that could work, as a software engineer, I don't feel comfortable enough needing a container with another distribution to do something that could be in my system.

In 2022 I started to work at SUSE and for the first time I tried the openSUSE distribution until today.

The Tumbleweed

Today I've three different computers with Tumbleweed running. One for work, Thinkpad T14s, one for personal usage, Dell inspiron 5490, and another one as a personal media server, Libre computer La-frite.

The best thing of having Tumbleweed for me is that I get the latest GNOME as soon as it's released. And another big thing for this distribution is how easy it's to fix something upstream thanks to the Open Build Service, but I work everyday with that, so I'm biased. For sure, any other community distribution has different ways to contribute, but I find this one easy enough.

Even being a rolling release distro, Tumbleweed doesn't break a lot. I can't say that it's stable, because the API of everything is broken everyday, but the distribution is tested for every release and at least some level of package compatibility check is done. That makes Tumbleweed a good distribution and I can update without fearing some weird package breakage.

I usually update my work and personal laptops once a week, and la-frite not so often, maybe every 6 months.

With the default installation, Tumbleweed uses btrfs with snapshots, and it's really easy to go back and forward using the snapper tool. So it's really easy to go back to a good state if the distribution is broken for some reason, and wait for a fix.

The problems that I found during this year

Some problems with the NVidia graphic card in my Dell laptop, some times the kernel and the driver were not working correctly. I had to use snapper to get the NVidia working again, but fixed a few days later.
Currently I'm having some random crashes because some bug with amdgpu and wayland and mutter, but it's not too annoying for me to go back, so I didn't use snapper this time and I'm facing this random crashes waiting for the fix.

Long live the Tumbleweed

So far so good. Tumbleweed is a nice distribution that I'm enjoying. It's not getting in the way and I can find almost anything that I need for work, programming, gaming, media, etc. I'm really happy with this distribution and it's the perfect distribution for people like me, that want to have the latest things.

I know that there are other openSUSE flavors that are interesting, like the immutable ones, Leap or the latest one Slowroll, but Tumbleweed is the one for me.