Speech compressor and limiter for your headset in Fedora

Last couple of months I’ve been using Google Hangouts and Bluejeans conferencing technologies more than my VoIP phone. I got used to crisp and clear voice from my Polycom and Platronics headset so I had a question.

Is it possible to achieve the same with my cheap USB Logitech headset which I use for this purpose? The obvious answer is – yes.

This is tutorial how to setup PulseAudio in Fedora 20+ with LADSPA plugins to improve audio experiences for conferences. It works best if you have a dedicated audio interface (e.g. USB headset) so you don’t need to play around with the configuration every time you start your conference. This setup is definitely not good for listening music!

What we need? Just couple of LADSPA audio plugins. Install them with:

yum -y install ladspa-swh-plugins

Our goal today is to improve volume with two well-known audio technique called signal compression. We will be using two plugins for that: a compressor and a limiter (which is technically also a compressor). Both are distributed in the package we have installed:

$ analyseplugin /usr/lib64/ladspa/dyson_compress_1403.so

Plugin Name: "Dyson compressor"
Plugin Label: "dysonCompress"
Plugin Unique ID: 1403
Maker: "Steve Harris <steve@plugin.org.uk>"
Copyright: "GPL"
Must Run Real-Time: No
Has activate() Function: Yes
Has deactivate() Function: No
Has run_adding() Function: Yes
Environment: Normal or Hard Real-Time
Ports:  "Peak limit (dB)" input, control, -30 to 0, default 0
        "Release time (s)" input, control, 0 to 1, default 0.25
        "Fast compression ratio" input, control, 0 to 1, default 0.5
        "Compression ratio" input, control, 0 to 1, default 0.5
        "Input" input, audio
        "Output" output, audio

$ analyseplugin /usr/lib64/ladspa/fast_lookahead_limiter_1913.so

Plugin Name: "Fast Lookahead limiter"
Plugin Label: "fastLookaheadLimiter"
Plugin Unique ID: 1913
Maker: "Steve Harris <steve@plugin.org.uk>"
Copyright: "GPL"
Must Run Real-Time: No
Has activate() Function: Yes
Has deactivate() Function: No
Has run_adding() Function: Yes
Environment: Normal or Hard Real-Time
Ports:  "Input gain (dB)" input, control, -20 to 20, default 0
        "Limit (dB)" input, control, -20 to 0, default 0
        "Release time (s)" input, control, 0.01 to 2, default 0.5075
        "Attenuation (dB)" output, control, 0 to 70
        "Input 1" input, audio
        "Input 2" input, audio
        "Output 1" output, audio
        "Output 2" output, audio
        "latency" output, control

Long story short, a compressor filter is able to gain volume of a signal in a dynamic way. Quiet portions of the stream will be boosted more than loud portions. This ratio can be configured as well as peak limit and release time of the limitation phase.

A limiter is basically high-ratio compressor which is used to hard-limit signal that is too loud. It is often used as the very last plugin to make sure you are getting the loudest signal possible, but not too high.

In other words, a person who talks quiet will be boosted on volume while loud conference attendees will be limited. That’s what we want.

In traditional setup, application sends the stream to the PulseAudio which sends it directly to the audio card:

Application -> PulseAudio -> Headset

We want to achieve the following setup:

Application -> PulseAudio -> Compressor -> Limiter -> Headset

With the LADSPA plugins we just installed this is a piece of cake. Before we start, we need to find out audio interface we want to “enhance”. In my case, this is the Logitech USB headset:

$ pacmd list-sinks | grep name:
name: <alsa_output.pci-0000_00_1b.0.analog-stereo>
name: <alsa_output.usb-Logitech_Logitech_USB_Headset-00-Headset.analog-stereo>

Now, to create those filters, just type in the following commands:

pacmd load-module module-ladspa-sink sink_name=ladspa_output.fast_lookahead_limiter_1913.fastLookaheadLimiter master=alsa_output.usb-Logitech_Logitech_USB_Headset-00-Headset.analog-stereo plugin=fast_lookahead_limiter_1913 label=fastLookaheadLimiter control=0,-10,0.25

pacmd load-module module-ladspa-sink sink_name=ladspa_output.dyson_compress_1403.dysonCompress master=ladspa_output.fast_lookahead_limiter_1913.fastLookaheadLimiter plugin=dyson_compress_1403 label=dysonCompress control=0,1,0.5,0.99

Note you need to make sure the master is set to the name of your output device (in my case the Logitech USB thing). That’s really it. Now, startup the PulseAudio mixer:

pavucontrol

Start up your conferencing software (or if it’s that a plugin in a browser) and redirect the output on the Playback tab to “Dyson Compressor”. You should immediately hear the difference - everything should be louder. You will likely hear more background noise - this is expected.

Having the mixed opened, head over to Output Devices to see how signal is being boosted and limited. You should see the input monitors on the two filter plugins and the output.

If you want to tune up parameters, just unload the module and try again:

pacmd list-modules
pacmd unload-module XYZ

When you are happy with this, put these two lines in the /etc/pulse/default.pa (skip the “pacmd” command) and you are good to go. PulseAudio will remember your setting per application, so you can route particular applications or plugins through this setup.

If you want to play more with this, there are LADSPA plugins for noise cancellation and equalizer to cut off low and high frequencies to clean out speech. Share your setup with me in the comments bellow.

EFI in QEMU KVM on Fedora 20

Last sprint, I was working on EFI PXE booting support for Foreman. Although I have a EFI-compatible PC in the house, I wanted a stable environment for development and testing. Virtualized, of course.

The best option is KVM/QEMU/libvirt triple which I use for regular Foreman development. I started googling around and spent a day modifying my distribution package. It did not work well. And then I stumbled upon familiar name, Laszlo Ersek from Red Hat, who helped me getting this rolling.

QEMU with EFI

To get EFI working under QEMU, one needs 2.1.1+ version with modified BIOS. Since I already almost destroyed configuration of the QEMU from Fedora 20 (had to reinstall the packages), I was guided to install from sources:

wget http://wiki.qemu-project.org/download/qemu-2.1.1.tar.bz2
tar bla bla
yum -y install spice-server-devel spice-protocol SDL-devel
./configure --target-list=x86_64-softmmu --enable-spice \
    --prefix=/opt/qemu-2.1.1 --enable-debug --disable-gtk
make install
chown root:root /opt/qemu-2.1.1/libexec/qemu-bridge-helper
chmod u=rwxs,g=rx,o=rx /opt/qemu-2.1.1/libexec/qemu-bridge-helper
echo 'allow virbr0' > /opt/qemu-2.1.1/etc/qemu/bridge.conf

Having stable version installed in a separate tree (which keeps your laptop clean along smile on your face), one needs to download and install BIOS and EFI firmware. This is important step - for PXE booting, latest nightly builds are needed. Fortunately, for Fedora/Red Hat users, there is a repo out there with latest and greatest build. Do not worry, those packages are compatible with Fedora and won’t overwrite/upgrade anything. You can install them next to the official QEMU BIOS:

sudo wget https://www.kraxel.org/repos/firmware.repo -O \
    /etc/yum.repos.d/kraxel-qemu-firmare.repo
yum install edk2.git-ovmf-x64

This guy installs into /usr/share/edk2.git along with some dependencies which are also compiled from git (SeaBIOS for legacy EFI mode, NIC drivers and so on).

Now, the hardest part. Since libvirt currently lacks UEFI support, we need to work directly with QEMU. I’d like to thank Laszlo for constructing me this script which sets up things correctly:

#!/bin/bash
OVMF_BINARY=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
VARSTORE_TEMPLATE=/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd
QEMU_ROOT=/opt/qemu-2.1.1

# create fresh variable store
cp $VARSTORE_TEMPLATE /tmp/guest-vars.fd

# run qemu
$QEMU_ROOT/bin/qemu-system-x86_64 \
  -M pc-i440fx-2.1 \
  -enable-kvm \
  -m 900 \
  -drive unit=0,if=pflash,format=raw,readonly,file=$OVMF_BINARY \
  -drive unit=1,if=pflash,format=raw,file=/tmp/guest-vars.fd \
  -global isa-debugcon.iobase=0x402 \
  -debugcon file:/tmp/guest.ovmf.log \
  -monitor stdio \
  -device piix3-usb-uhci -device usb-tablet \
  -netdev bridge,id=net0,br=virbr0,helper=$QEMU_ROOT/libexec/qemu-bridge-helper \
  -device virtio-net-pci,netdev=net0,romfile=,bootindex=0 \
  -device qxl-vga

You can run this script as regular user the only part requiring root access is the networking one (bridge helper) and that has suid bit set. The VM will be connected to the “default” libvirt virtual NAT network (virbr0) where you can set up your PXE environment and start playing with EFI PXE booting.

PXELinux and Foreman

In this second part of my blog entry, I want to share some news about Foreman support. Since Foreman ships with templates set for PXELinux by default, I wanted to keep on this path.

Apparently syslinux 5.x does not have EFI support and syslinux 6.02 did not work for me. So I decided to compile 6.03 from sources which did not work either. Luckily, folks on the IRC channel recommended to use 6.03-pre20 version which worked like a charm:

https://www.kernel.org/pub/linux/utils/boot/syslinux/Testing/6.03/syslinux-6.03-pre20.tar.gz

The following files need to be extracted into tftpboot/efi64 directory:

syslinux.efi
chain.c32
ldlinux.e64
libutil.c32
menu.c32

Also I created relative symlinks (TFTP runs in chroot usually) for configuration and kernels:

boot -> ../boot
pxelinux.cfg -> ../pxelinux.cfg

You can do the same with 32bit arch. But that’s pretty much it!

DHCP daemon configuration is straighforward and the best approach is only to hand over EFI PXELinux to DHCP clients which reports as EFI-compatible. You need something like (192.168.100.0 network with Foreman running on .2):

# ... snippet ...

option arch code 93 = unsigned integer 16;

subnet 192.168.100.0 netmask 255.255.255.0 {
    pool {
        range 192.168.100.10 192.168.100.200;
    }

    option subnet-mask 255.255.255.0;
    option routers 192.168.100.1;

    class "pxeclients" {
        match if substring (option vendor-class-identifier, 0, 9) = "PXEClient";
        next-server 192.168.100.2;

        if option arch = 00:07 {
            filename "efi/syslinux.efi";
        } else if option arch = 00:06 {
            filename "efi32/syslinux.efi";
        } else {
            filename "pxelinux/pxelinux.0";
        }
    }
}

Similarly you can use Grub, you need to copy EFI binary and create configuration file. This is covered in RHEL6 (1) and RHEL7 (2) documentation.

Tunnel into your libvirt NAT network with SOCKS

My development setup is based on two Fedora boxes: laptop and worstation situated in Red Hat office. Both machines are powerful enough to run few VMs, but to leverage hardware, I wanted to use libvirt virtual networks on both devices. The Xeon server is more capable of running nested KVM virtual appliances like oVirt or RHEV while laptop is good for local testing.

I have two virtual (NAT) libvirt networks called:

• zzz.lan (server)
• local.lan (laptop)

My goal was to use them seamlessly during my development. Let’s focus on the local.lan first.

Some time ago I was using libvirt bridges, but since I need to tackle with DHCP/TFTP/DNS services (I work on The Foreman project), NAT was the only viable option here. The configuration is pretty standard, I created new NAT network and disabled DHCP service there because I wanted to run my own DHCP server. You can use “default” one for this purpose.

The key thing is to setup local caching DNS server dnsmasq. In my case, I use Google public DNS servers as main forwarders:

laptop# cat /etc/dnsmasq.d/caching
bind-interfaces
listen-address=127.0.0.1
resolv-file=/etc/resolv.dnsmasq

laptop# cat /etc/resolv.dnsmasq
nameserver 8.8.8.8
nameserver 8.8.4.4

laptop# cat /etc/resolv.conf
nameserver 127.0.0.1
domain redhat.com

laptop# chattr +i /etc/resolv.conf

Having dnsmasq set correctly, now we can add special file that will add all hosts that has been created via libvirt. This is done automatically by libvirt.

laptop# cat /etc/dnsmasq.d/local.lan
addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts

Restart (or start and enable) the service.

laptop# systemctl enable dnsmasq
laptop# systemctl start dnsmasq

Tip: When using VPN you can add extra configuration file so all *.redhat.com queries goes to internal DNS servers (these IP addresses are not real). Also we want to use public DNS server for the adress of the VPN hub, otherwise you would need to use IP address for that service.

laptop# cat /etc/dnsmasq.d/redhat.com
server=/redhat.com/10.20.30.40
server=/redhat.com/10.20.30.50
server=/vpn-hub1.redhat.com/8.8.8.8

Now, everytime you create a new VM locally, you only need to refresh dnsmasq configuration to re-load the default.addnhosts file.

laptop# virt-install --os-variant rhel6 --vcpus 1 --ram 900 \
    --name myhost.local.lan --boot network --nodisk --noautoconsole

This is as easy as sending HUP signal (or you can do service reload) or similar:

laptop# pkill -SIGHUP dnsmasq

That’s it. You can connect to your server easily:

laptop# ssh root@myhost.local.lan

I do more. When you shutdown or restart your VM, libvirt (which uses dnsmasq for DHCP too) can assign different IP, therefore the addnhost entry is not valid anymore. Therefore I created a simple script that preallocates static DHCP entry in libvirt, so restarts won’t hurt anymore. The snippet is something like:

echo "Removing existing DHCP/DNS configuration from libvirt"
netdump="sudo virsh net-dumpxml default"
virsh_dhcp=$($netdump | xmllint --xpath "/network/ip/dhcp/host[@mac='$MAC']" - 2>/dev/null)
virsh_dns=$($netdump | xmllint --xpath "/network/dns/host/hostname[text()='$FQDN']/parent::host" - 2>/dev/null)
sudo virsh net-update default delete ip-dhcp-host --xml "$virsh_dhcp" --live --config 2>/dev/null
sudo virsh net-update default delete dns-host --xml "$virsh_dns" --live --config 2>/dev/null

while true; do
  AIP="192.168.100.$(( ( RANDOM % 250 )  + 2 ))"
  echo "Checking if random IP $AIP is not in use"
  $netdump | xmllint --xpath "/network/ip/dhcp/host[@ip='$AIP']" - &>/dev/null || break
done

echo "Deploying DHCP/DNS configuration via libvirt for $AIP"
sudo virsh net-update default add-last ip-dhcp-host --xml "<host mac='$MAC' name='$FQDN' ip='$AIP'/>" --live --config
sudo virsh net-update default add-last dns-host --xml "<host ip='$AIP'><hostname>$FQDN</hostname></host>"  --live --configpice,listen=0.0.0.0 -

This will remove existing entries, generates random IP address, checks if that does not exist and create DHCP (and in addition to that DNS) entries.

Now, this was the easy part. I use the same on my server, but I want to be able to access the zzz.lan VMs from my laptop too. For web access this turns out to be quite easy task. For example in Chrome you only need to created this file:

laptop# cat ~/proxies.pac
function FindProxyForURL(url, host) {
  if (shExpMatch(host, "*.zzz.lan*")) {
    return "SOCKS5 localhost:8890";
  } else {
    return "DIRECT";
  }
}

Start dynamic tunnel to the libvirt hypervisor (the server):

laptop# cat .ssh/config
Host server
    HostName server.redhat.com
    DynamicForward 8890

laptop# ssh -N server &>/dev/null &

And start Chrome with this configuration:

laptop# google-chrome --proxy-pac-url=file:///home/lzap/proxies.pac

When working on The Foreman development, I also want the application to be able to connect to various services (like oVirt or RHEV compute resources). This turns to be possible too. The key utility is tsocks wrapper which is available in Fedora. We can use the very same ssh dynamic tunnel for that.

laptop# cat /etc/tsocks.conf
local = 192.168.0.0/255.255.255.0
server = 127.0.0.1
server_port = 8890

The only difference in dnsmasq configuration on the server is that I want to open it for incoming DNS requests, because my laptop needs to resolve hosts from zzz.lan:

server# cat /etc/dnsmasq.d/caching
bind-interfaces
resolv-file=/etc/resolv.dnsmasq
interface=em1
interface=lo
no-dhcp-interface=em1

And of course we need to open firewall port:

server# firewall-cmd --add-service=dns --permanent

Now on the laptop, one more change is needed. I need to direct my local dnsmasq daemon to resolve from the server dnsmasq (assuming my server IP address is 10.90.90.90):

laptop# cat /etc/dnsmasq.d/zzz.lan
server=/zzz.lan/10.90.90.90

After reloading dnsmasq, I can finally use tsocks:

laptop# tsocks wget -O - http://ovirt34.zzz.lan

So I can start my application allowing it to connect to zzz.lan NAT network:

laptop# tsocks foreman_application

That’s it. Hope you were inspired.

Track short lived processes with auditd

I had a hard time to find some more information about short lived processes when I was building Foreman SELinux policy for OpenStack Installer (Staypuft). I wanted to see process name and all the parameters for all possible processes spawned on my system. My plan was to write a SystemTap script, but thank to Mirek Grepl and Steve Grubb I learned much easier trick to do that. This assumes you have auditd daemon installed (RHEL/Fedoras are fine):

auditctl -a exit,always -S execve

Then tail your /var/log/audit/audit.log to see messages. Warning: This can flood your system heavily. You will see some decent content there like:

type=CWD msg=audit(1401202071.000:14728):  cwd="/"
type=PATH msg=audit(1401202071.000:14728): item=0 name="/usr/sbin/ps" nametype=UNKNOWN
type=SYSCALL msg=audit(1401202071.000:14729): arch=c000003e syscall=59 success=yes exit=0 a0=7fe10bbe6369 a1=7fe10bbe7580 a2=7fff0b1d3208 a3=0 items=2 ppid=26400 pid=5161 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=875 comm="ps" exe="/bin/ps" subj=unconfined_u:system_r:passenger_t:s0 key=(null)
type=EXECVE msg=audit(1401202071.000:14729): argc=5 a0="ps" a1="-o" a2="pid,ppid,%cpu,rss,vsize,pgid,command" a3="-p" a4="26974,26817"

I was able to tell that mod_passenger is spawning /bin/ps every few seconds. This is very useful when tracking down some processes with long puppet runs. To turn the audit rule down, replace -a option with -d:

auditctl -d exit,always -S execve

You can filter out audit logs, see auditctl manual page. For example I was interested in processes running in passenger_t SELinux domain:

auditctl -a exit,always -S execve -F subj_type=passenger_t

To add rules permanently, edit /etc/audit/audit.rules file and restart auditd daemon.

SystemTap as a system wide strace tool

I needed to find a process that was searching for several file patterns. I could use strace tool but since this one was a Apache2 module, I would need to hack startup scripts and probably create a wrapper script. There was another way of doing that - using SystemTap.

My problem was simple - I wanted to see process name, pid and full absolute path for a filename pattern. The goal was set, with little bit of googling, I was able to write this (not ideal and hacky) SystemTap script:

$ cat syscall_open.stp
#!/usr/bin/env stap
#
# System-wide strace-like tool for catching file open syscalls.
#
# Usage: stap syscall_open filename_regexp
#
global myfilename
probe syscall.open {
  myfilename = filename
}
probe syscall.open.return {
  if (myfilename =~ @1) {
    printf ("%s(%d) opened %s -> %d\n", execname(), pid(), myfilename, $return)
  }
}

Before we can use SystemTap, we need to install those packages (this is for RHEL6/CentOS6). Note you do need debug info packages, otherwise this will not work (enable them via subscription-manager or in repo files):

$ yum -y install systemtap systemtap-runtime kernel-debuginfo-`uname -r` \
    kernel-debuginfo-common-`uname -i`-`uname -r` kernel-devel-`uname -r`

Usage is super simple, you can either make it executable or to see more verbose output just do:

$ stap -v syscall_open.stp native_support.so

This is effective for the whole system, after Apache2 httpd server restart and first access, I was able to find why the heck mod_passenger searches on wrong paths:

ruby(4257) opened "/usr/lib/ruby/gems/1.8/gems/passenger-4.0.18/lib/native/passenger_native_support.so" -> -2
ruby(4257) opened "/opt/rh/ruby193/root/usr/local/share/ruby/site_ruby/native/passenger_native_support.so" -> -2
ruby(4257) opened "/opt/rh/ruby193/root/usr/local/lib64/ruby/site_ruby/native/passenger_native_support.so" -> -2
ruby(4257) opened "/opt/rh/ruby193/root/usr/share/ruby/vendor_ruby/native/passenger_native_support.so" -> -2
ruby(4257) opened "/opt/rh/ruby193/root/usr/lib64/ruby/vendor_ruby/native/passenger_native_support.so" -> -2
ruby(4257) opened "/opt/rh/ruby193/root/usr/share/rubygems/native/passenger_native_support.so" -> -2
ruby(4257) opened "/opt/rh/ruby193/root/usr/share/ruby/native/passenger_native_support.so" -> -2
ruby(4257) opened "/opt/rh/ruby193/root/usr/lib64/ruby/native/passenger_native_support.so" -> -2
ruby(4257) opened "/usr/share/foreman/.passenger/native_support/4.0.18/ruby-1.9.3-x86_64-linux/passenger_native_support.so" -> -2
ruby(4257) opened "/usr/share/foreman/.passenger/native_support/4.0.18/ruby-1.9.3-x86_64-linux/passenger_native_support.so.rb" -> -2
ruby(4257) opened "/usr/share/foreman/.passenger/native_support/4.0.18/ruby-1.9.3-x86_64-linux/passenger_native_support.so.so" -> -2
ruby(4282) opened "/usr/lib/ruby/gems/1.8/gems/passenger-4.0.18/lib/native/passenger_native_support.so" -> -2
ruby(4282) opened "/opt/rh/ruby193/root/usr/local/share/ruby/site_ruby/native/passenger_native_support.so" -> -2
ruby(4282) opened "/opt/rh/ruby193/root/usr/local/lib64/ruby/site_ruby/native/passenger_native_support.so" -> -2
ruby(4282) opened "/opt/rh/ruby193/root/usr/share/ruby/vendor_ruby/native/passenger_native_support.so" -> -2
ruby(4282) opened "/opt/rh/ruby193/root/usr/lib64/ruby/vendor_ruby/native/passenger_native_support.so" -> -2
ruby(4282) opened "/opt/rh/ruby193/root/usr/share/rubygems/native/passenger_native_support.so" -> -2
ruby(4282) opened "/opt/rh/ruby193/root/usr/share/ruby/native/passenger_native_support.so" -> -2
ruby(4282) opened "/opt/rh/ruby193/root/usr/lib64/ruby/native/passenger_native_support.so" -> -2
ruby(4282) opened "/usr/share/foreman/.passenger/native_support/4.0.18/ruby-1.9.3-x86_64-linux/passenger_native_support.so" -> -2
ruby(4282) opened "/usr/share/foreman/.passenger/native_support/4.0.18/ruby-1.9.3-x86_64-linux/passenger_native_support.so.rb" -> -2
ruby(4282) opened "/usr/share/foreman/.passenger/native_support/4.0.18/ruby-1.9.3-x86_64-linux/passenger_native_support.so.so" -> -2

SystemTap is awesome tool. It helps me every week :-)