Upload your site to Netlify using their incremental deployment API

I've recently switched to a setup where I do all my builds for this blog on Travis. While doing so I needed to migrate away from using Netlify's internal build infrastructure. This resulted in a quick python script that allows you to upload arbitrary directory tree to Netlify and does so using their incremental deployment API. All that means that while this site is quite big in size the deployments go rather quickly ! There are some known issues but apart from them the script should just work for any custom Netlify deployment you would like to have. I use it on this very site, to have a preview of any PR before merging it as well as for deploying the main site after the PR is merged. I hope you will find it useful and please do not hesitate if you want to post an issue or a PR !

Running host programs in isolation inside one-off Docker containers

I am quite bad at remembering how to launch docker to have everything set up correctly. Hence the following - a script that launches any commandline specified in its arguments inside a new docker container. Current directory is mounted inside the container automatically, so the thing you are executing can have its local dependencies satisfied.

CURRENT_DIRECTORY=`pwd -P` # untangle symbolic links if needed - SELinux needs the real path

if [[ -z $1 ]]; then
    echo "usage: `basename $0` command_to_run_inside_a_container"
    exit 1

docker run -i -t -v "$CURRENT_DIRECTORY":"$MOUNT_PATH":Z $IMAGE bash -c "useradd -M -d '$MOUNT_PATH' $USERNAME && cd '$MOUNT_PATH' && bash -c '$RESOLVED_ARGUMENTS'"

# restore SELinux context for the current directory
restorecon_path=`which restorecon`
if [[ -x "$restorecon_path" ]]; then
    restorecon -R "$CURRENT_DIRECTORY"

I use vanilla Debian Jessie as a run platform there, mostly because this is what most of my servers run. The script covers setting up SELinux and mounting the directory from which it is run as /mnt inside the container while also having the default non-root user added.

Run Jessie, run !

Using ad hoc Docker volumes on SELinux systems

I've recently tried running some quick Docker commands using host's directory as a volume:

docker run -i -t -v `pwd`:/mnt debian:jessie bash -c "ls -hal /mnt"
ls: cannot open directory /mnt: Permission denied

I use Fedora as my main OS, which, it turns out, has some pretty nice SELinux settings. These deny access from inside the container to the outside. Said Fedora consists mostly of almost-newest-but-stable everything though, which makes Docker to be in a fairly recent version. A version that understands how to change a SELinux context for the directory we're mounting, by itself ! You need at least Docker v1.7 for this.

docker run -i -t -v `pwd`:/mnt:Z debian:jessie bash -c "ls -hal /mnt"
total 8.0K
drwxrwxr-x.  2 1000 1000 4.0K Dec 30 18:34 .
drwxr-xr-x. 21 root root  242 Dec 30 19:07 ..

Please notice the capital Z as a mount parameter. And that is it. Mostly. Some cleanup remains, as docker leaves the host's directory with a changed SELinux context. To restore it you need to

restorecon -R `pwd`

Or use any other path you'd like instead of `pwd` in the examples above. Happy dockerizing !

RustFest - organization was the best. Also rhymes.

I went to RustFest and it was amazing !

It was clearly the best conference organizational-wise I've been to so far. It made me think of what I really liked there. What made it so awesome and welcomy ? To me this was a large number of small things just coming together. I listing them here for me to remember and for others to use. Let's make conferences at least this friendly, together !

Before the event:

  • very clear emails, repeating messages couple of times in different emails for the important stuff
  • maps of the popular routes provided, e.g. from the airport and bus station
  • supporter ticket sale - two times the price - you pay for another person that wouldn't be able to get a ticket otherwise
  • survey on dietary requirements/allergies
  • survey on childcare needs
  • clear statement of the code of conduct for the conference

During the event:

  • very good MC person

    • keeping people entertained through the tech breaks
    • keeping tabs on the talk length, allowing appropriate amount of questions if the time allowed
  • live captioning of all talks - small screen outside of the view of the main screen with the text live

  • getting the next speaker prepared and mic-tested before the end of the current talk

  • quiet room to rest with clear rules on no talking and interrupting there

  • clear signage for the different parts of the venue (washrooms, quiet room, party space etc)

  • washrooms

    • all same, ungendered

    • basic items for free in said washrooms

      • chewing gum
      • tooth paste
      • tampons, pads
      • baby wipes
  • info desk/registration

    • clear info during registration, handing out programme
    • asking people if they want to be on the photos - giving out lanyards accordingly - flashy red lanyards for people who do not want photos of them taken
  • emergency number to call with stated purpose

    • code of conduct violation
    • if you are lost in the city

The best terminal emulator for Windows 10's Bash or how to run X applications

update as of 03/2017
As this article is by far the most popular on my site right now I feel that an update is necessary, as the landscape evolved a bit since the original publish date.
Conemu started to support Bash on Windows properly now and this is what I settled on in the end.
The article below is still relevant for the running X applications on Windows part though.

I've been playing a bit with GNU/kWindows a.k.a. Bash on Windows a.k.a Windows Subsystem for Linux (Beta). I was especially interested whether I can use my regular Linux dotfiles to recreate my working environment of zsh + tmux + vim. The biggest troubles I had were with the terminal emulator. While the default one, invoked by saying bash is much better than powershell or cmd.exe already - it lacks some things I've learned to rely on, like 256-colour palette support. Thus the search for the ultimate terminal emulator begun. I tried ConEmu, cmder and their spinoffs to no avail. Either the colours were lacking, or the emulator would swallow up certain strokes like the arrow keys. Then I thought - hey, I use terminator on Linux, maybe it would be possible to use it here as well. To my surprise the answer was - yes !

  • install Windows Subsystem for Linux
  • restart Windows
  • install VcXsrv
  • run VcXsrv
  • invoke the following from the bash console:
sudo apt-get install terminator
DISPLAY=:0 terminator -u
  • profit !


The font rendering is not ideal and the borderless mode does not work, but hey, it is quite usable nonetheless ! It even has bidirectional clipboard sharing with Windows' windows, which is good. P.S. konsole and gnome-terminal do seem to have troubles launching (crashy-crashes there)

Don't throw away. Fix.

Try fixing your things. It's already broken - no need to sweat.
You might learn something.
Level 2 - push for things to be easily fixable.
It's good for you. It's good for the planet.

image0 image1 image2 image3 image4

Conferences you did not know existed and now you know you just need to go to

Go to conferences. It's good for you.

Ever wondered why to go to conferences ? Is it worth it, especially if you do not feel confident that the conference is on something that would be in an immediate realm of your day-to-day interests ? Just go ! You might end up learning something and meeting new people. And new people mean more variables to mix up your life. A bit scary, I know. Still worth it though. Wondered what conference in Europe are weird enough so people wouldn't mind having another misfit ? Here you go.


Now imagine something niche within the industry. To me, AI comes to mind. Mages in robes in their towers, sending out autonomous robot armies against each other. Right ? There's a conference for that ! At nucl.ai, in July, in Vienna, you can meet AI professionals from academia and industry alike, most of them coming from game development background. The atmosphere is very newcomer-friendly, the only requirement is that you're interested in AI, not necessarily having anything to do with it professionally. Petra and Alex, the organizers, are wonderful people that make this conference quite unique. It's happy fun to see ideas floating from academia to the industry and vice versa. Don't forget the presentations on work-in-progress major AAA games ! I've learned there that the everyday programming in the games industry is quite different than the one I do in terms of best practices and patterns. Definitely worth a try !

Erlang User Conference

Speaking of something completely different. Erlang ! Was functional before being functional was considered cool. Very practical, very fast, parallelize everything ! Processes everywhere ! Message passing ! Want to feel the groove as well ? Visit Stockholm in September for Erlang User Conference. Meet creators of the language, talk about massive scale deployments. Other FP languages are welcome. I would even say, the more strange the better. Idris seems to be the topic of interest as well.  Eat good food, walk the city of dynamite. Erlang has a quite close-knit community which gives you this warm family feeling.


Speaking of family. There's a family of conferences known as SoCraTes. Software Crafstmanship and Testing. SoCraTes. Get it ? :D As it's more than one conference a year, it's harder to go to all of them. I only went to Codefreeze, in Finland, in January, and only once, but loved it already. Codefreeze, Finland, in January. Get it ?! :D First thing is that you don't know what will happen there. You just go, meet the people and then you sit down and everyone talks about what they would like to learn and what they can share. This way you come up with some loose agenda you follow more or less. One track, very intense. Talking, swimming in ice water, learning, sauna going. Very intimate feel, not many people, lots of things to learn.

Chaos Communication Congress

Speaking of lots of people and intimate feel. Chaos Communication Congress is, I think, the biggest gathering of all nerds and freaks in Europe, yet being there feels quite at home. The best organized conference I've been to. Friendly people showing their wares on their humble tables, 4 simultaneous tracks, huge building with intricate hallways. It's cyberpunk, it's inclusive, it's about hardcore infosec topics, it's about politics. Everything's there, everyone's there. 4 days of data being constantly uploaded to your brain, on the waves of Club Mate. Feels like an extended holiday present, as it's happening in the last week of December.

The Camps

Chaos Communication Camp and The Dutch Camp. Imagine CCC, the Congress, but in the woods. Middle of nowhere, multigigabit internet connection. sleeping in the tent, then going to a world-class lecture. Both CCC, the camp, and The Dutch Camp, the camp, are happening every 4 years, each of them taking alternating slots. So every 2 years there is a camp somewhere. Did I mention that the location changes every event ? Happy camping !


So go out there. Explore. Meet people. Learn. Have fun. None of the above are much expensive and most of the events have a policy of "write to us if you want to go but can't afford the ticket". Friendly people fixing problems together.

nucl.ai: July, tickets around 250EUR
EUC: September, tickets around 350EUR
SoCraTes: the cost and the time depend on the event
CCC: last week of December, tickets around 100EUR
The Camps: summer, tickets around 125EUR

Productive New Year's Eve

It turns out that the last day of the year can be quite productive. I've managed to sneak in a PCB layout for the input measurement and protection board for my sunpowered server . Got it ordered, can't wait to test it ! I got carried away with via stitching there a bit - we'll see how hard it will be to solder the larger components because of that. The whole project is hosted here - it's open hardware so please do take a look.

image0 image1 image2

Sporting Let's Encrypt certificate

Let's Encrypt is soon to be publicly available service to provide TLS certificates for free and in automated fashion. I'm helping them test their systems by participating in the beta program. Because of that this site is now sporting, publicly recognizable certificate from Let's Encrypt. Yay ! The work they are doing I believe is of much importance. Lack of easy path for getting TLS certificate was a major hurdle for the most of the website owners, otherwise keen on getting their traffic encrypted. This, in turn, is important for two reasons. Not allowing the content you are viewing on the internet to be snooped at, not leaking you private data is one thing. Other, less known, but as much important, is countering the ability of injecting any malicious code/data into the responses from the servers you got and in turn preventing lots of man in the middle attacks. Hopefully, when Let's Encrypt becomes publicly available we would be on the path to much more friendly internet for all.

Fixing timeouts running Android integration tests

The main problem with Android CI I had recently is that, after switching to Lollipop, the integration tests wouldn't run. Invoking androidConnectedTest gradle target always resulted in crashing with ShellCommandUnresponsiveException. Internet says that in such a case ou just need to set ADB_INSTALL_TIMEOUT. Set and then nothing. Sourcediving it is then !

A long while after that I got to this file: Device.java [Linking to master, here's the commit hash:1cb1a4c2976b99ae53d28d7f01d975232c85f990, as I don't seem to be able to find how to link to that hash directly] What do we see there ? That indeed ADB_INSTALL_TIMEOUT is being used:

static {
    String installTimeout = System.getenv("ADB_INSTALL_TIMEOUT");
    long time = 4;
    if (installTimeout != null) {
        try {
            time = Long.parseLong(installTimeout);
        } catch (NumberFormatException e) {
            // use default value

So far so good, ADB_INSTALL_TIMEOUT system variable seems to be respected when invoking package installation tools. Are the above the only methods that can install a package though ? Going further on that hunch we see that in addition to installing single packages there is a possibility of having a multi-package installation session.

public void installPackages(List<String> apkFilePaths, int timeOutInMs, boolean reinstall, String... extraArgs) throws InstallException {
if (getApiLevel() < 21) {
        Log.w("Internal error : installPackages invoked with device < 21 for %s",Joiner.on(",").join(apkFilePaths));
        if (apkFilePaths.size() == 1) {
                installPackage(apkFilePaths.get(0), reinstall, extraArgs);
        Log.e("Internal error : installPackages invoked with device < 21 for multiple APK : %s", Joiner.on(",").join(apkFilePaths));
        throw new InstallException("Internal error : installPackages invoked with device < 21 for multiple APK : " + Joiner.on(",").join(apkFilePaths));
String sessionId = createMultiInstallSession(apkFilePaths, extraArgsList, reinstall);

Aha ! No-Lollipop check here, with a fallback to the old method - we may be onto something ! Some lines pass and we can see an invocation of createMultiInstallSession. What's there ?

private String createMultiInstallSession(List<String> apkFileNames, @NonNull Collection<String> extraArgs, boolean reinstall) throws TimeoutException, AdbCommandRejectedException, ShellCommandUnresponsiveException, IOException {
String cmd = String.format("pm install-create %1$s -S %2$d", parameters.toString(), totalFileSize);
executeShellCommand(cmd, receiver, DdmPreferences.getTimeOut());

A different invocation of executeShellCommand, now using DdmPreferences.getTimeOut() as a timeout value source. Summarizing - this only happens if you install multiple applications for your androidConnectedTest and you are using android device to test on that has api version that is equal or greater to 21. That is all cool that we had this little Computer Science Investigation, but how to fix that - i.e. how to have proper timeouts for your installations ? Ideally from somewhere you configure and/or invoke your builds. It turns out that gradle supports invoking just enough Java for us to use there. In your gradle.build as the very first lines:

println "setting global timeout for apk installation to 10 minutes"

android {
        compileSdkVersion compileSdk
        buildToolsVersion buildTools

That's it. Invoke your android tests with ADB_INSTALL_TIMEOUT env variable set AND have the DddPreference set in your gradle.build as in the example above and you should be golden. Happy droiding !