Graph the noise level in your office in 15 minutes

This is a recurrent complaint in any open space: “There is too much noise!” (the other one is that the climate is too cold/too hot). There are some usual culprits, but it is nice to have data to back your complaints up.

I will here show you how to generate a real-time noise level graph in 15 minutes without any material beside our laptop or desktop, not even a microphone is needed. This is a dirty hack, but it works and can be put in place very quickly with just a few command lines. The steps, which will be mostly cut&paste are:

  • install a noise recorder tool
  • set up nginx to serve the data recorded
  • use a nice javascript library to display the data properly

I used soundmeter, a python tool. So first, install it:

# make sure we can install python packages
apt-get install virtualenv
# install required dependencies for building soundmeter
apt-get install python-dev portaudio19-dev python-dev alsa-utils
# install other useful tools for later display
apt-get install nginx expect-dev
# set up a directory for the tool
mkdir $HOME/soundmeter
# create virtualenv
virtualenv $HOME/soundmeter
# activate it
source $HOME/soundmeter/bin/activate
# install soundmeter
pip install soundmeter

Et voilà, your recorder is setup.

But do you not need a microphone? Well, either you have a laptop with a build in microphone, either you can just plug an headphone, which is basically a microphone used the other way (produce instead of record sound).

To get data, a one-liner is enough:

soundmeter --segment 2 --log /dev/stdout 2>/dev/null | unbuffer -p perl -p -e 's/\s*(\d+)\s+(.{19})(.*)/"$2,". 20*log($1)/e' > meter.csv

The explanation is as follow:

  • soundmeter: run soundmeter forever (–seconds to limit the duration)
  • --segment 2: output data every 2 seconds (default 0.5 seconds, but is very spiky)
  • --log /dev/stdout: default data on stdout is not useful for graphing, we need to log to a file. Use /dev/stdout as file to actually log to stdout
  • 2>/dev/null: do not pollute output
  • |: the output is not in a great format, it needs to be reformatted
  • unbuffer -p: by default data is buffered, which is annoying for real-time view. This does what the name suggests
  • perl -p -e: yummy, a perl regexp!
  • s///e: this will be a substitution, where the replacement part is a perl expression
  • \s*(\d+)\s+(.{19})(.*): record value and timestamp stripped of the milliseconds
  • “$2,”: display first the timestamp with a comma for csv format
  • 20*log($1): the values from soundmeter are in rms, transform them in dB via the formula 20 * log (rms)
  • > meter.csv: save data in a file

In short, we do the following transformation on the fly and write it to a csv file:

2015-09-22 13:36:13,082 12 => 21.5836249,2015-09-22 13:36:13

You now have a nice csv file. How to display it? Via a nice html page with the help of a javascript library, dygraphs,of course.

Set up nginx by adding in /etc/sites-enabled/noise the following content (replace YOUR_HOME by your actual home directory, of course):

server {
 listen 80;
 root YOUR_HOME/soundmeter;

and restart nginx:

service nginx restart

Then setup you page in $HOME/soundmeter/noise.html:

<script src="//"></script>

#graphdiv2 { position: absolute; left: 50px; right: 10px; top: 50px; bottom: 10px; }

<div id="graphdiv2"></div>
<script type="text/javascript">
 g2 = new Dygraph(
 "http://localhost/meter.csv", // path to CSV file
 delimiter: ",",
 labels: ["Date", "Noise level"],
 title: ["Noise (in dB)"],
 showRoller: true,

You can of course replace localhost by your IP to publish this page to your colleagues.

Now just go to http://localhost/noise.html:


Indispensable tool of the day: thefuck

Thefuck is a magnificent app which corrects your previous console command.

Thefuck is a python tool, hosted on github, which looks at your previous command and tries to correct it. You of course invoke it after a typo for instance, by typing fuck in your terminal.

A few examples in image, from the developer himself:

thefuck in action

A few more examples:

Adds sudo for you:

~> apt-get install thefuck 
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
~> fuck
sudo apt-get install thefuck [enter/↑/↓/ctrl+c]
Reading package lists... Done

Corrects your git branch syntax (this is my main use of thefuck):

~/gits/lomignet-apt> git push
fatal: The current branch newfeature has no upstream branch.
To push the current branch and set the remote as upstream, use

  git push --set-upstream origin newfeature

~/gits/lomignet-apt> fuck
git push --set-upstream origin newfeature [enter/↑/↓/ctrl+c]
Total 0 (delta 0), reused 0 (delta 0)
 * [new branch] newfeature -> newfeature
Branch newfeature set up to track remote branch newfeature from origin.

The installation is trivial as the package is uploaded to the python package index:

pip install thefuck

To then have the command available, add to your .bashrc or whichever your startup script is:

eval $(thefuck --alias)

In the background, thefuck has a set of known rules which you can find in the readme. If those rules are not comprehensive enough for you, you can write your own.

Restart Mongodb with not enough nodes in a replica set

The context could be a virtualised cluster, where an hypervisor went suddenly down. 2 of your Mongo replicas are unavailable, only 1 is left, which then of course drops back to being secondary and read only.

You want to have this server running alone for a while while the others come back online, as you decide that it is better to have potential small inconsistency instead of not running for a few hours. The thing is that this last server will complain that the rest of the set is not available. To get it started again, you just need to make it forget about the rest of the set.

  1. Switch the service off
    service mongodb stop
  2. Remove the line replSet from your /etc/mongodb.conf
  3. Restart the service
    service mongodb start

    Mongo will complain:

    mongod started without --replSet yet 1 documents are present in local.system.replset
     [initandlisten] ** Restart with --replSet unless you are doing maintenance and no other clients are connected.
     [initandlisten] ** The TTL collection monitor will not start because of this.
     [initandlisten] ** For more info see
  4. Remove the offending document in system.replset from the mongoshell
    // will give you one document back
    // remove all documents (there is only one)
    // check resultset is empty
  5. Restart mongo
    service mongodb stop
    service mongodb start
  6. Once the other nodes are up, add again the replSet line in /etc/mongodb.conf and restart the service.

New puppet apt module, now with better hiera!

The puppet apt module from puppetlabs works great, but has one big issue. You can define sources (and keys, settings and ppas) from hiera, but only your most specific definition will be used by the module, as only the default priority lookup is done. This means a lot of cut & paste if you want to manage apt settings across multiple environments or server roles. This is known, but will not be fixed as it is apparently by design.

Well, this design did not fit me, so I forked the puppetlabs module, updated it to enable proper hiera_hash look up, and published it to the puppet forge. There is no more difference with the original, but it does simplify my life a lot. Now if you define multiple sources in your hierarchy, for instance at datacenter level:

   location: ''
   release: '%{::lsbdistcodename}'
   repos: 'main contrib non-free'

and at server level:

    location: ''
    repos: 'main'
      id: '47B320EB4C7C375AA9DAE1A01054B7A24BD6EC30'
      server: ''

you will nicely have both sources in sources.list.d, instead of only having the one defined at server level.

You can find the source on github, and can download the module from the puppet forge. Installing it is as simple as:

puppet module install lomignet-apt

Better sudo messages

There is a nice feature to increase the “usefulness” of your error message when a user enters a wrong password.

Just add to /etc/sudoers the following line:

Defaults insult

et voilà:

admin@localhost:~$ sudo ls
[sudo] password for admin: 
I can't hear you -- I'm using the scrambler.
[sudo] password for admin: 
Have you considered trying to match wits with a rutabaga?
[sudo] password for admin: 
Wrong! You cheating scum!
[sudo] password for admin: 
You type like I drive.
[sudo] password for admin: 
Your mind just hasn't been the same since the electro-shock, has it?
[sudo] password for admin: 
stty: unknown mode: doofus
[sudo] password for admin: 
Listen, burrito brains, I don't have time to listen to this trash.
[sudo] password for admin: 
You speak an infinite deal of nothing
[sudo] password for admin: 
That's something I cannot allow to happen.
[sudo] password for admin: 
I feel much better now.
[sudo] password for admin: 
He has fallen in the water!
[sudo] password for admin: 
I don't wish to know that.

The only thing if you set sudo up that way is that you should expect a big surge in security logs once you users discover it and start playing with it.

Writing your first Sublime Text 3 plugin

Sublime text is an amazing text editor. Sleek, full of features, multi platform, very usable without a mouse. It is my editor of choice since a few years already.

One of its great advantages is that it is extensible in python, which makes it very easy to tweak.

I recently played with a vagrant box in which I needed to update a file. The file was mounted inside vagrant, but needed to be copied elsewhere inside the box, meaning it had to be copied manually every time I saved it. As I am very lazy (laziness drives progress, one of the favorite saying of my director of studies) I wanted to do that automagically. This is a very simple job, ideal for a first plugin.

So, where to start? Well, it is quite easy: on the menu bar click tools, then new plugin, Et voilà, you have your first plugin. Congratulations!

import sublime, sublime_plugin
class ExampleCommand(sublime_plugin.TextCommand):
 def run(self, edit):
 self.view.insert(edit, 0, "Hello, World")

This is a nice skeleton, but it does no go very far.

As I want to have an action on save, I needed to have an event listener plugin, in my case listening to a save event. I wanted to act after the event (use the file only after it was written). The API says that on_post_save_async is the best event for me, as it runs in a separate thread and is non blocking.

import sublime
import sublime_plugin

class UpdateOnSave(sublime_plugin.EventListener):

    def on_post_save_async(self, view):
        filename = view.file_name()
        # do something with the filename

Good, this is getting somewhere! All the base is present, now I just had to do something with this file.

The something is that case was a kind of subprocess call, to use vagrant ssh. Sublime already has a wrapper around subprocess, named exec. Exec can run in 2 contexts, view (basically the sublime buffer you are editing) to run TextCommands or window (sublime itself) to run all type of commands. Finding in which context to run your command is a bit of trial and error, but once done the last bit of the plugin is thus a trivial (once you know it) call to exec:

# please always use shlex with subprocess
import shlex
import sublime
import sublime_plugin
import os

class UpdateOnSave(sublime_plugin.EventListener):

    def on_post_save_async(self, view):
        filename = view.file_name()
        savedfile = os.path.basename(filename)
        saveddir = os.path.dirname(filename)

        # write in sublime status buffer
        sublime.status_message('Manually saving ' + filename)

        source_in_vagrant = '/vagrant/' + savedfile
        dest_in_vagrant = '/project/' + savedfile

        cmd_cp = "vagrant ssh -c 'sudo cp {0} {1}'".format(source, dest)

        view.window().run_command('exec', {
            'cmd': shlex.split(cmd_cp),
            'working_dir': saveddir,

Good, your plugin is ready! The last question is to know where to put it to have it actually used. With Sublime 3 under Linux, you need to have it in $HOME/.config/sublime-text-3/Packages/User. Note that it must be named, with the .py extension, not the .py3 extension, or it would not be found.

You can find the plugin on github.

Puppet error messages and solutions

This is a collection of error messages I got while setting up a puppet infrastructure and testing modules, as well as their reasons ans solutions.

Failed to load library ‘msgpack’


On an agent:

Debug: Failed to load library 'msgpack' for feature 'msgpack'
Debug: Puppet::Network::Format[msgpack]: feature msgpack is missing

on the master:

Debug: Puppet::Network::Format[msgpack]: feature msgpack is missing
Debug: file_metadata supports formats: pson b64_zlib_yaml yaml raw


This happens when you run puppet agent, for instance.


msgpack is an efficient serialisation format. Puppet uses is (experimentally) when communicating between master and agent. This format requires a gem, which if not installed will give this debug message. This is completely harmless, it just pollutes your logs.


Just install the msgpack ruby gem. Depending on your system, you can

#debian based:
apt-get install ruby-msgpack
gem install msgpack

This immediately removes the debug messages. To actually use msgpack, you need to add in the [main] or [agent] section of puppet.conf the line:

preferred_serialization_format =  msgpack

Could not retrieve information from environment


Error: /File[/var/lib/puppet/facts.d]: Could not evaluate: Could not retrieve information from environment $yourenvironment source(s) puppet://localhost/pluginfacts


Puppet agent run.


If no module has a facts.d folder, puppet will throw this error. This is an actual bug in puppet, at least version 3.7.3.


Option 1: Just discard. This is shown as an error, but has no impact and the run will carry on uninterrupted.

Option 2: actually create a facts.d folder in a module.

Could not find data item classes in any Hiera data file


Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find data item classes in any Hiera data file and no default supplied on node


Your puppet tests were all working fine from vagrant. You just installed a puppet master and the first agent run gives you this error.


Check your hiera.yaml file (in /etc/puppet/hiera.yaml, /etc/puppetlabs/puppet/hiera.yaml or pointed by hiera_config from your puppet.conf). There is a :datadir section, telling puppet where to find hiera data. If the path there is absolute, then it should directly point to the directory. If it is relative, then it works only under vagrant and is based on puppet.working_dir.


Many options are possible.

  • Use a common absolute path everywhere.
  • Put the directory, maybe via a link, in its default location.
  • Puppet can interpolate variables when reading datadir, so if your issue is due to different environments, you could use a path like

Note that if you change hiera.yaml, you need to reload the puppet master as hiera.yaml is only read at startup.

No such file or directory @ dir_s_rmdir


Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed when searching for node $nodename: No such file or directory @ dir_s_rmdir - /var/puppet/hiera/node/$nodename.yaml20150812-5415-1802nxn.lock


Puppet agent run


  1. Puppet master tries to create a file in a directory he does not own, and has thus no permission.
  2. Puppet tries to create a file or directory whereas the parent does not exists.
  3. The partition where puppet tries to create a lock file is full.


  1. With the path given in the example error message:
    chown -R puppet:puppet /var/puppet/hiera
  2. Make sure the parent is created in the manifest as well

This ‘if’ statement is not productive


 This 'if' statement is not productive.

followed by some more explnation depending on the context.


Puppet agent run


Puppet does not want to leave alone, and pretends to know better than me. I might want to have a if (false) {…} or if (condition) {empty block} for whatever reasons, but no, puppet very violently and rudely bails out. There is a bug discussion about it as well as a fix to change the wording, but the behaviour will stay.


Comment out what puppet does not like.

sslv3 alert certificate revoked or certificate verify failed


SSL_connect returned=1 errno=0 state=SSLv3 read server session ticket A: sslv3 alert certificate revoked


Wrapped exception:
SSL_connect returned=1 errno=0 state=unknown state: certificate verify failed: [self signed certificate in certificate chain for /CN=Puppet CA:]


Puppet agent run


You probably revoked or clean certificates on the puppet master, but did not inform the agent about it. or maybe you are now pointing to a new puppetmaster.


You can fix this by cleaning the agent as well:

sudo rm -r /etc/puppet/ssl
sudo rm -r /var/lib/puppet/ssl