Веселый носорог

Equivalent of C# delegates in Python

Originally published at www.ikriv.com. Please leave any comments there.

Coming from C# background, I was trying to understand whether Python allows to pass obj.method where ordinary function is expected, and if yes, how could it possibly work.

Consider this code fragment:

def call_twice(f):

class Foo:
    def __init__(self, data):
        self.data = data
    def print_value(self):
        print(f"Value is {self.data}")

foo = Foo(42)
call_twice(foo.print_value)  # can it work? how?

The short answer is: yes, it works, foo.print_value returns an equivalent of C# delegate that knows about foo variable.

Longer answer

Unlike C++ and the languages influenced by it, methods definitions in Python accept “this” parameter explicitly. It is usually called “self” by convention, but in fact any name would work:

class Foo:
  def print_value(this):
     print(f"Value is {this.data}")

It looks suspiciously like C, and in C passing a one-argument function to something that expects a zero-argument function would not work. So, how can we pass foo.print_value to call_twice() and then invoke it with no arguments?

The answer is that while Foo.print_value is indeed an ordinary function that accepts one argument, foo.print_value is a callable object that accepts no arguments. When invoked, it calls Foo.print_value(foo). This is very similar to how C# delegates work, and somewhat similar to C++ functional objects that define operator ().

In other words, foo.print_value() is not simply syntactic sugar for Foo.print_value(foo) as I originally thought. The expression foo.print_value returns an object of type method. This object can then be called immediately, or passed to another function to be called later.

More details here: https://stackoverflow.com/questions/6685232/how-are-methods-classmethod-and-staticmethod-implemented-in-python

Some Interesting Implications

Since methods are actually regular functions, they can be applied to arbitrary objects. Consider this code:

class Bar:
    def run(self):

    def go(self, msg):

class Bus:
    def __init__(self, name):
        self.name = name

    def go(self, msg):
        print(f"{self.name} is going {msg}!")

bus = Bus("Yellow bus")
Bar.run(bus)  # prints "Yellow bus is going fast!"

We apply Bar.run() to an object of type Bus, which is completely unrelated to Bar, and yet it invokes Bus.go() and prints the expected message. Of course, this feature should be used with care, but it does occasionally become useful, e.g. when we want to be 100% certain method of what class we are calling on an object in presence of multiple inheritance.

Веселый носорог

The growth of command-line options: 1979-2017

Originally published at www.ikriv.com. Please leave any comments there.

I stumbled upon an article (https://danluu.com/cli-complexity/), that explores the dramatic growth of the number of command line options in most popular Unix commands between 1979 and 2017 and explains why it may have happened. I found it very interesting and instructive to read.

I am personally a big fan of “do one thing and do it well” phylosophy, but I concur that it does not always work. E.g. I am really grateful for grep -r for recursive “find in files” option, so I don’t have to write something like find . -type f | xargs grep.

I was also unpleasantly surprised by behavior of git blame mentioned in the article: it turns out that git blame -C -C means “dig deeper in history than git blame -C, and git blame -C -C -C means “dig even deeper”. This is not how options usually work in Unix, I would expect something like git blame -C 2 or git blame -C --deep.

Веселый носорог

“You made 5 billionth search” scam

Originally published at www.ikriv.com. Please leave any comments there.

When I was going to Worldometer’s coronavirus page, it would redirect me to some stupid scam web site saying “you made 5 billionth search”. It happened only one one of my devices, the one given to me by my employeer, and the one I least expected to be infected, since I use it strictly for work purposes.

It turns out nothing was wrong with the device: it’s a targeted ad on the site itself that did it. The solution was to change my Google Ads ID in the settings (Settings -> Privacy -> Ads -> Reset advertising Id on Android).

Веселый носорог

Python: logical operations with None and Booleans

Originally published at www.ikriv.com. Please leave any comments there.

Coming back to the blog after long break.

Today, almost by accident I found how logical operators work with None in Python. The result is surprising, because it is not commutative.

(False or None)  is None
(None  or False) is False
(True  or None)  is True
(None  or True)  is True

(False and None)  is False
(None  and False) is None
(True  and None)  is None
(None  and True)  is None

There seems to be little logic here (pun intended), but in fact it all makes sense. For “or”, if the first operand is truthy, it is returned, and the second operand is not calculated. If the first operant is falsy, the second operand is returned. For “and”, if the first operand is falsy, it is returned, and if it is truthy, the second operand is returned. The definitions are not symmetrical, which results in non-commutative behavior with None.

UPDATE: Javascript behaves the same way with null>. E.g. (null && false) === null, but (false && null) === false.

Веселый носорог

Can’t reuse Dell computer parts

Originally published at www.ikriv.com. Please leave any comments there.

I have a very nicely built 12 year old Dell tower, that I inherited from some startup, which I use as a home server running Ubuntu. I also have a 7 year old spare CPU and motherboard with some memory on it. The tower was quite high end in its time and is very nicely built: sturdy case, huge fans, 750W power supply, but the hardware inside is, of course, quite outdated. I thought of removing the 12-year old motherboard and CPU and replacing them with much faster 7 year old motherboard and CPU, but had no luck.

Where most standard cases use a loose assortment of wires for power, reset, HDD led, and the like, Dell uses a combined 40-pin connector that integrates the entire front panel, including two USB slots. This may be easier for installation and what-not, but it means regular non-Dell motherboard cannot be installed in this case, it cannot accept the 40-pin connector, and thus cannot be powered up, which, of course, is critical.

I thought of perhaps buying a new case and reusing the power supply, but alas: the power supply H750P is also non standard and not compatible with regular ATX motherboards.

Way to go Dell! The whole point of IBM PC was compatibility of components. I hate it when companies start messing with standards in order to ensure vendor lockdown (I am looking at you, Apple!). Didn’t really help Dell though.

Oh, well, I guess I will have to get by with the old hardware for now, it is doing alright, and then at some point I will have to buy a new case and a new power supply, even though these ones could still be perfectly working, they were built to last.

Веселый носорог

Guido said it would cause too much disturbance…

Originally published at www.ikriv.com. Please leave any comments there.

Python has two ways to convert an object to a string: str(x) and repr(x). str is supposed to be user-readable, and repr is more technical, e.g. for debugging purposes.

To my surprise, if I have a list of things, str(list) and repr(list) do the same thing and include a repr of each item. I would expect str(list) to include str of the list items, and repr(list) to include repr of the items.

Apparently, I am not alone: back in 2008 this was proposed by Oleg Broytman and Jim J. Jewett (PEP 3140), but rejected, because “Guido said it would cause too much disturbance too close to Beta“. It is causing disturbance ever since, I suppose…

Веселый носорог

Type cast in Python

Originally published at www.ikriv.com. Please leave any comments there.

Python type cast looks like an assertion:

# Python
x = expression_returning_Base
assert isinstance(x, Derived)

This corresponds to Java or C#

// Java
Derived x = (Derived)expression_returning_Base;

In languages like Java and C++, it is a common situation when at runtime we know we have an object of type Derived, but the compiler thinks it is of type Base, so we must explicitly cast the object reference to type Derived before the compiler would allow us to use any Derived-specific properties or methods.

Python, being a dynamic type language, does not have this problem: you just call whatever methods you feel right. If such methods are not found at runtime, an exception will occur.

# Python
class Base:
  def foo(self) -> None:

class Derived(Base):
  def bar(self) -> None:

def func() -> Base:
  return Derived()

x = func()
x.bar() # it works, but fails the type check

However, the situation changes in presence of type hints: x.bar() above won’t pass the type check, claiming that "Base" has no attribute "bar". This does not prevent us from running the code, but may, say, prevent from committing it to a repo if type checks are strictly enforced.

If we simply declare x of type Derived, it does not help:

x : Derived = func() # typecheck error

The error is Incompatible types in assignment (expression has type "Base", variable has type "Derived").

To convince the type checker that x is indeed of type Derived, we need an assertion:

x = func() 
assert isinstance(x, Derived) # type cast

Note, that the situation may become complicated if you apply type assertion to attributes. Unlike C++ or Java, type checking is not part of the language runtime, so whether the assert will work depends on the type checker.

x = type('test', (), {})() # empty object of custom type
x.y = func()
assert isinstance(x.y, Derived)
x.y.bar() # May or may not pass the type check

Specifically, current version of mypy recognizes this assert, but pyre does not.

Веселый носорог

Bash: is missing file older than existing file?

Originally published at www.ikriv.com. Please leave any comments there.

In bash it is possible to test whether one file is older or newer than the other:

if [ file1 -nt file2 ]; then
# do something

if [ file3 -ot file4 ]; then
# do something

However, the man page does not say what happens if one of the files does not exist. Experiment shows that at least on my systems (Ubuntu 16.04 and 18.04) non-existing files are consistently treated as created in the infinite past. I.e., any existing file is newer than non existing file, and any non existing file is older than any existing file. The results are summarized in the table below:

[ existing -nt missing ] ==> TRUE
[ missing -nt existing ] ==> FALSE
[ existing -ot missing ] ==> FALSE
[ missing -ot existing ] ==> TRUE
[ missing -nt missing ] ==> FALSE
[ missing -ot missing ] ==> FALSE

BTW, here’s the script that prints this table:


function compare {
  echo -n "[ $1 $2 $3 ] ==> "
  if [ $1 $2 $3 ]; then echo TRUE; else echo FALSE; fi

touch existing
compare existing -nt missing
compare missing -nt existing

compare existing -ot missing
compare missing -ot existing

compare missing -nt missing
compare missing -ot missing
Веселый носорог

Docker: why one process per container does not always work

Originally published at www.ikriv.com. Please leave any comments there.

My “permanent” SSL certificates are expiring soon, so I decided to switch to letsecnrypt. The easiest way to obtain their certificate is allegedly to use Certbot.

There is a nice docker image for Certbot, but even the authors caution against using it, unless you really know what you’re doing.

When certbot runs inside the web server’s container, it can do everything automatically. When it runs outside, a lot of things must be done by hand, but the biggest problem is fighting over who controls port 80, and web server restarts.

Certbot needs to listen to incoming HTTP requests in order to prove to letsencrypt that you own the domain you claim. When it runs inside, it modifies the server config and ensures proper files are served, so everything just works. When it runs outside, it cannot modify the config.

If we want to keep the main web site running while certbot is working, we somehow need to proxy incoming HTTP requests to the certbot. This is not very hard if the certbot were in a constantly running container, but in reality it is executed via docker run --rm command, so its container is short lived. Proxying such a thing is probably possible, but not easy.

Without the proxy, we must shut down the web server container while certbot is running, but it would mean hard downtime for the web site users. If one merely changes the config, most web servers know how to do graceful restart without refusing incoming connections, but if the web server container is stopped, no requests will be served until it is up again.

So, as with the mail server, practical considerations forced me to give up on the “pure” solution and bake certbot into my web server container.