Add Self-Signed Certificates to your local trusted CA

When you use different self hosted services, such as your own Gitlab instance for example, you probably use self signed certificates for securing the connection. Self-signed certificates have the disadvantage that Browsers and other applications do not trust them and therefore either display error messages or refuse to connect at all. The IDEA IntelliJ IDE for instance does not clone from repositories which are protected by self signed certificates. Instead of disabling the SSL certificate verification completely, it is recommended to add your self signed certificate to the local certificate store.

Tracking Changes in Latex – The Easy Way

Latex has many advantages in comparison to WYSIWYG word processors such as Microsoft Word or Libre Office. Being a text based markup language, Latex allows distraction free writing in a large selection of editors, it scales easily also for large projects with hundreds or thousands of references and images within the text and it produces very clean layouts, which can be exchanged simply by using different document classes. If you use a version control system such as git or svn, also versioning becomes very convenient, every change can be seen in the text file as a simple diff.

Data Wrangling with csvkit and SQLite

As mentioned earlier, csvkitis a very convenient tool for handling coma separated text files, especially when they are too large to be processed with conventional spread sheet applications like Excel or Libre Office Calc. The limits of Office programs can rather easy be reached, especially when dealing with scientific data. Open Office Calc supports the following limits. maximum number of rows: 1,048,576 maximum number of columns: 1,024 maximum number of sheets: 256 Excel offersalso 1 048 576 rows but provides 16,384 columns.

Create an ER Diagram of an Existing SQLite Database (or many other RDBMS)

Visualisation helps solving problems and is therefore an important tool database design. Many database providers have their product specific tools for re-engineering existing schemata, but self-contained, serverless, embedded relational database management systems(RDBMS) such as SQLiteoften come without much tooling support. The extremely small footprint of SQLiteprovides a very powerful tool for implementing database driven applications without the hassle of database administration, user privilege management and other demanding tasks that come with more complex systems.

Importing your Repositories from Github to your Gitlab Instance

Github is a great service and offers a free micro accountfor students, which includes five private repositories. Still this might not be enough private repositories for some of us, especially if you start a lot of side projects (which are hardly finished :-)) and want to keep your code and data organised, versioned and collaborative but non-public (because it is not finished yet :-)). For this and other reasons I am also running a private Gitlabinstance, which basically offers the same features as Github and can be hosted on your private server.

A quick fix for the ‘ascii’ codec can’t encode character error in Python and the csvkit tool

For a current project I need to migrate large volumes of CSV data into a relational database management system. The Python driven csvkit is the swiss army knifeof CSV tools and very handyfor this purpose. However, a few CSV files cause troubles when I tried to pipe the SQL CREATE statements I created with the csvkit tool to a file. csvsql -i sqlite -d ';' -e 'utf8' --db-schema test_schema --table test_table inputfile.

Den UPC Wireless Cable Router durch den eigenen WLAN-Router ersetzen

Nachdem das Netzteil meines UPC Modems das zeitliche gesegnet hat, pilgerte ich zum UPC Store am Keplerplatz und ließ das gesamte Gerät tauschen. Der Austausch erfolgte unkompliziert, doch leider hatte ich anstelle des althergebrachten Modems nun endgültig einen Wifi-Router erhalten, wogegen ich mich bisher immer gesträubt hatte. Schließlich betreibe ich einen liebevoll eingerichteten Netgear WNR 3500L mit der altenativen Firmware TomatoUSB und hätte keine Lust, mein Setup auf dem UPC Gerät umzusetzen und auf Funktionen zu verzichten.

Migrating Markup Languages

There exists a broad range of different markup languages, which allow you to structure, highlight and format your documents nicely. Different wiki platforms utilise various markup languages for rendering texts, Github and other code sharing platforms use their own dedicated languages for their documentation files. These markup languages such as Textile or Markdown work rather similar, as they provide users with a basic set formats and structural elements for text. In order to avoid vendor lock-in or when you need to move to a different system, you will need to extract the information from the old markup and make it available in a different language.

Neue Rezension: iX 11/2015

Im iX Magazinfür professionelle Informationstechnik wurde unser Buch in der Ausgabe 11/2015 von Bernhard Röhrig rezensiert. Gelobt wurden fachliche Qualität, Umfang, Testdatenbank und Layout, Kritik gab es für das Verstreuen der Neuerungen sowie kleineren Detailangaben zu speziellen Features.

Backup your Emails or Migrate between two IMAP Servers with Offlineimap

Syncing IMAP Accounts Between Servers Due to a change in our mail server infrastructure, I had to move all emails from the old account to a new IMAP account. Obviously if you create the new account in your email client such as thunderbird, you can simply drag and drop the messages between the accounts. Unfortunately, timeouts and other errors prevent you from being sure that all emails have been moved between the servers correctly, if you rely on the mail client only.

Blattlinien in überregionalen Tages- und Wochenzeitungen

Österreich verfügt über ein überschaubaren Zeitungsmarkt, dem es insbesondere an überregionalen Qualitätserzeugnissen mangelt. Trotz einiger weniger Lichtblicke, beherrscht das Kleinformat den Markt und damit zu weiten Teilen die öffentliche Meinung. Um dem Zeitungssterben entgegenzuwirken, bieten die meisten Tages- und Wochenzeitungen zusätzlich ein Online-Portal an. Das Österreichische Mediengesetz sieht in periodisch erscheinenden Medien eine sogenannte Offenlegung (§25) vor, in der nicht nur die Eigentumsverhältnisse der Medien, sondern auch eine Erklärung über die grundlegende Richtung eines periodischen Druckwerks anzugeben ist.

Bypassing Blocked Ports in (Public) Wifi Networks

Wifi networks at conferences or hotels often block certain ports, either for enhancing security or simply by misconfiguration. Blocking ports can be used for preventing some applications from accessing the internet via their access points. Other usages can be to force users to use encrypted protocols, instead of their clear text alternatives. In order to make this process easier, in many cases the ports which should be allowed are whitelisted, whereas everything else is blocked.

Neue Rezension: Linux Magazin – Alles über MySQL

Das Linux Magazin hat in seiner Ausgabe 09/2015 die folgende Rezension veröffentlicht: Dass dieses Buch wenige Fragen offen lassen will, machen schon seine Dimensionen deutlich: Über 800 Seiten ergeben einen soliden Wälzer. So stellen die Autoren ihren umfangreichen Erklärungen zu MySQL denn auch eine Einführung in das Thema Datenbanken an sich voran und machen den Leser mit den grundlegenden Techniken der Datenmodellierung bekannt. Im zweiten Teil geht es dann systematisch um die MySQL-Administration: Er erläutert die Architektur der Datenbank, die Installation und die Bordwerkzeuge nebst einem Abstecher zu den Themen Backup und Monitoring sowie die Performance-Optimierung im Hinblick auf Hardware und Konfiguration.

Die dritte Auflage des MySQL Handbuchs

Nachdem wir im Dezember erfahren hatten, dass sich die Bestände der zweiten Auflage zu Ende neigen, ging es für uns Autoren an die Planung der dritten, aktualisierten Auflage des MySQL Buchs. Von Jänner bis April 2015 haben Eva Zangerle, Wolfgang Gasslerund ichan der Neuauflage unseres Buches gearbeitet. Eva, Wolfgang und ich haben seitenweise Release-Notes durchgeackert, Leser-feedback und wünsche eingearbeitet und natürlich so gut wie alle neuen Features berücksichtigt. Selbstverständlich haben wir auch das Datenset komplett neu erzeugt und mit Millionen von Einträgen gefüttert.

Persist Web Radio Streams in the Pi MusicBox

After upgrading my Raspberry Pi Model B to the new Raspberry Pi 2 a few weeks a go, I finally found a new purpose for the old device. There exist several images for the Pi with a focus on with Internet radio streaming. After doing a little research, I decided to use the Pi MusicBox, which comes with a headless audio player and supports a whole lot of different sources and formats: Spotify, SoundCloud, Google Music, Podcasts (with iTunes, gPodder directories), local and networked music files (MP3/OGG/FLAC/AAC), Webradio (with TuneIn, Dirble, AudioAddict, Soma FM directories), Subsonic.

Update OwnCloud’s Admin UserID

After I installed OwnCloud on a local server, I realized that my choice of the administrative username was ambiguous and that I would never be able to remember it. The interface from OwnCloud allows you to change the full name of a user, but not the login name (user id) from a user via its interface. As OwnCloud stores its complete configuration in a database, I decided to update the information directly in the MySQL instance.

There was always snow at Christmas – Retrieving and processing historical weather data with Forecast IO

The weather was unusually mild this winter and many people seemed to remember back to the days of their childhood. In our memories, there was always snow at Christmas, but is that actually true? The curious among us know what to do, so lets see what the Internet has to offer for verifying our hypothesis. Unfortunately there are hardly any open weather data sources allowing to retrieve the weather conditions for any historical date., but in this short blog post I would like to show you how you can combine a few APIs and a little Java programming in order to retrieve the appropriate data. The complete project can be downloaded at GitHub.

Debugging Thunderbird Problems with Linux

Sometimes it can be hard to reproduce problems related to Email as many factors can be a cause for the trouble. Thunderbird (and all other Mozilla products) provide logging facilities which allow tracing the activities of the application and to detect the source of an error. The logging mechanism is activated by setting two environment variables as described in the documentation. The following two lines will set the environment export NSPR_LOG_MODULES=imap:2,smpt:3,timestamp export NSPR_LOG_FILE=/tmp/mail.

Build a SMS notification service

Automated event notification is useful for many applications. In a recent blog post, I presented a solution how to use Twitter for notifying about server events. In this post you learn how to achieve the same by using an SMS API. The rationale behind an SMS service is its robust availability. There exist plenty of different services, I chose ClockworkSMS, as it provides a simple interface for several programming languages, plugins and a prepaid billing service.

Using Twitter to get the current IP address of your Raspberry Pi

The Raspbery Pican act as a rather neat server due to its low power consumption and its flexibility. As it is very small and can be powered with a standard dell phone charger, its convenient to take it with you. But running in headless mode (i.e. no monitor is attached) and without a keyboard connected it can be a pain to “find” the device in a network as it receives a dynamic IP address whenever it gets connected via ethernet or wifi.

Use a TP-Link TL-WR710N Router as a Repeater with OpenWRT

The TL-WR710N offers five modes of operation: Router mode Client mode (connects ethernet devices to an existing wifi) Repeater mode (extends existing wifis) Access point mode (create a wifi hotspot from a wired connection) WISP Client Unfortunately the firmware which is installed on the device is very instable and barely usable for my purpose. I could not get connectivity by using the software which is pre-installed although it would offer a nice interface for all the mentioned modes.

Virtuelle Maschinen – Die perfekte Testumgebung

Wer sich richtig mit MySQL auseinandersetzen möchte muss natürlich alles ausprobieren, an jeder Schraube drehen und jede Detaileinstellung testen. Da intensive Tests nicht an einem Produktivsystem durchgeführt werden sollten, bietet es sich an auf eine virtuelle Umgebung auszuweichen, die vom eigentlichen System abgekapselt ist. Eine solche Testumgebung lässt sich sehr leicht mit der kostenlosen Software VirtualBox realisieren. Eine genaue Anleitung dazu habe ich im Blog zum MySQL-Buchverfasst.

Configuring a catch-all VirtualHost for Apache2

Recently I noticed an interesting behavior of my Apache2 setup when I introduced a new VirtualHost and enabled the site. Usually I store the configuration for each host in an individual file, because it is much easier to maintain. The main page that I serve is this WordPress blog, having the ServerName blog.stefanproell.at and several aliases. The configuration files have the same name as the ServerName of the VirtualHost they configure.

Upgrading Samsung Ultra Series 5 Notebook with a SSD

During the holidays I finally found the time to upgrade my Samsung notebook by replacing the standard hard disk with a much faster SSD. There are plenty of tutorials available and the complete process is actually quite easy. For copying (and backing up) the data I used a docking station that can handle all kinds of disks, which is quite convenient for moving data between the two hard drives. My current setup consists of a “emergency” Windows 7 installation for anything which does not run with Linux and my main Linux Mate setup on a separate partition.

MySQL on DigitalOcean Servers Crashing

The smalles instance of a DigitalOcean droplet only provides 512GB of RAM. Do not forget that this rather low in comparison with other Web servers. If you do not check your configuration files for Apache and MySQL and leave them at defaults, you can easily run out of memory, because DO droplets come without swap by default. In the case of MySQL, your will be confronted with this error message: