Skip to content

2020

  • 1 min read

How this Blog Works internally

Just a few notes how my blog works internally.

Flask Server

The blog backend is written in Flask, a rather minimalist web framework in Python. As there are no users here, I don't need any authorization. I do not have a database backend. Anything I need is available via Web APIs.

Markdown

I write my posts in Markdown. To parse them and display them as HTML, I use the markdown2 Python package, which supports a few extras compared to Vanilla markdown, but is also still rather basic. Still, the parser is very fast and reliable.

Some Custom Stuff

I have added a few functionalities, e.g.

  • LaTeX formulas are rendered with QuickLaTeX (see the corresponding blog post for this)
  • Bibliographic references are rendered using my BibSonomy collection of papers. BibSonomy offers quite a few export formats which makes it ideal as a centralized paper repository (see e.g. how I use it in LaTeX)

GitLab CI

Publishing new posts works like a charm thanks to GitLab's awesome CI functionalities. Every time I push to my blog repository, it automatically deploys everything to my webspace and restarts the web service. This way, I do not need any edit functionalities in my blog, can quickly experiment with new features on my development machine (without breaking production) and always have a backup and history. If that isn't awesome, nothing is.

  • 1 min read

Marrying BibSonomy and BibLaTeX

(This is a rewrite of an older blog post of mine that disappeared together with that awful WordPress page)

I manage my scientific references in BibSonomy. With a standard LaTeX+BibTeX setup, I would have to download my bibliography and save it into a file. When doing this again for every paper, I sooner or later end up with many versions of my BibTeX entries, since I introduce some abbreviations in order to squeeze the whole thing into some random page limit, or maybe I correct some things, etc. etc.

Managing my references in BibSonomy should however help me to centralize them and be consistent throughout all of my works. This is where BibLaTeX (and biber) comes into play. Using BibLaTeX, I am able to not only provide a file as input for BibTeX, but also a web URL. Luckily, BibSonomy allows us to export a reference list as bibtex using a simple webcall. For example, if I want to export the list of publications that I tagged with myown (which can be found at https://www.bibsonomy.org/user/thoni/myown) as a BibTeX file, I simply add a /bib directly after the base URL: https://www.bibsonomy.org/bib/user/thoni/myown.

Finally, I enter the following command into my LaTeX main file:

\addbibresource[location=remote]{https://www.bibsonomy.org/bib/user/thoni/myown}

and poof, I have access to all my standardized references that I worked so hard to assemble.

Have fun writing your paper, thesis, commentary... :)

  • 1 min read

GitLab's CI rocks

In fact, I love it so much that I use it to automagically update this blog by simply pushing to its Git repo!

What is CI?

CI stands for "continuous integration" and describes the process of developing software inside an infrastructure ecosystem instead of for that infrastructure ecosystem.

Why is GitLab's CI so cool?

I'm maybe a bit biased here, but GitLab's CI is integrated directly inside GitLab.

This means

  • No extra software needed (compared to Jenkins)
  • Easy configuration inside a project/repository via a simple yml file with intuitive commands
  • Extensive documentation
  • Broad variants of GitLab's build tool, the GitLab runner (e.g. simple bash execution, execution via SSH, in a Docker image or even in a K8s cluster)
  • Automagically activated
  • Direct and useful feedback inside the GUI
  • 2 min read

Python3: exec in global vs local scope

Today, I encountered a seemingly strange problem. When using exec to execute a command that you do not want to implement hard into your code, everything works fine and as expected in a global scope. That means, if you execute the exec command in the global part of a script or hack it into your favourite CLI, the following code behaves as expected.

>>> import pandas as pd
>>> df = pd.DataFrame({"a": [1,2,3,4], "b":[4,3,2,1]})
>>> print(df)
   a  b
0  1  4
1  2  3
2  3  2
3  4  1
>>> exec('df=df[df["b"] > 2]')
>>> print(df)
   a  b
0  1  4
1  2  3

The exec command evaluates the filter expression and assigns its value to df again. So far, so good.

However, if you now move the exec command to a local scope, i.e. inside a function, it suddenly ceases to yield the expected result.

>>> import pandas as pd
>>> def lol(df):
...   exec('df=df[df["b"] > 2]')
...   return df
>>> df = pd.DataFrame({"a": [1,2,3,4], "b":[4,3,2,1]})
>>> print(df)
   a  b
0  1  4
1  2  3
2  3  2
3  4  1
>>> print(lol(df))
   a  b
0  1  4
1  2  3
2  3  2
3  4  1

Obviously, I cannot overwrite the df variable here. On the other hand, it is however perfectly possible to create new variables in the local scope:

>>> def some_method():
...     print(locals())
...     exec("never_before_seen_variable = 5")
...     print(locals())
...
>>> some_method()
{}
{'never_before_seen_variable': 5}

As a side note: It generally is not a good idea to use exec anywhere in your code as it can and will introduce unforeseen and unwanted side effects. For example, the code above will not fail since the variable assignment is a perfectly valid expression. However, it does not have the desired effect (to be fair, it does not have any effect). Additionally, exec can execute arbitrary code and whole code blocks. If you don't know exactly who will be using your code (and as a consequence, will have access to your exec statement), better leave it out.