Perl in Python - bring syntax into your code

The Zen of Python encourages us to write clean code, and there are techniques available to help you make ugly code beautiful - decorators, metaclasses, even monkeypatching can be used to shift complexity away from the code you're writing and leave your logic clear and uncluttered.

But there is a trade-off - by doing this you will increase the complexity of your code elsewhere - sometimes quite significantly. This approach is therefore not something we use often in our client work, but it can be good fit for library code - especially code intended for release.

I gave a talk about this at PyCon 2019 - you can see the full talk on YouTube.

However, I also used one of my personal projects as an example of how you can take things too far - that just because you can doesn't mean you should.

Perl in Python

I introduced my new Perl module for Python - bringing the elegance of Perl's native regular expression syntax into your daily python code:

$ pip install perl
$ python
>>> import perl
>>> value = "Hello there"
>>> if value =~ /^hello (.+?)$/i:
...     print("Found greeting:", $1)
...
Found greeting: there
>>> value =~ s/there/world/
>>> print(value)
Hello world

This works and is up on PyPI now - it essentially monkeypatches the Python interpreter to add Perl regular expression syntax to the language.

It is, by any measure, an abomination - an exercise in what you definitely shouldn't do, even though Python lets you.

It is full of horrors such as injecting Python's regular expression module into every imported file:

import re
builtins.__dict__["re"] = re

It also uses import hooks to pre-parse the source as it's imported so that it can be rewritten into Python before the real interpreter sees it:

$foo = "snakes are great"
$foo =~ s/snake/camel/i

# becomes

__perl__var__foo = "snakes are great"
__perl__var__foo = __perl__reset_vars() or re.sub(
r'snake', 'camel', __perl__var__foo, count=1, flags=re.I,
)

That lets us replace variables and rewrite entire lines of code on the fly, which is about as sensible as it sounds. The cost of introducing this becomes obvious as soon as an error is raised in the code we've generated - suddenly your error messages don't make any sense without an in-depth knowledge of how the module works:

>>> print($1)
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name '__perl__var__1' is not defined

Now, there are ways around this which will be coming in future versions of perl - we can use an excepthook to rewrite the error message back to what was originally written - but that will be after a few more features have been introduced. Next up is compulsory semicolons at the end of lines.

On a related note, do let me know if you can help write a PEP to get something into Python 4...

---

This article is based on my talk at PyCon UK 2019: