Improved the blog theme

I’ve improved the navbar, it should now work on mobile devices. Thanks for reporting this.

I tried to update to Bootstrap 3 but it did not work well. I guess I will stay on the old version for now. Christmas time!

02 December 2016 | blog

XTerm Zen mode

I do lot of talks these days with some amount of shell live demos. Although it is easy to configure xterm for beamer (black on white, big font, fullscreen), I created an alias for that:

xterm -cm -fa Monospace -fs 32 -bg white -fg black -cr red -bc \
    -bcf 200 -bcn 200 -xrm "allowColorOps false" -fullscreen \
    -e "PS1='$ ' LC_ALL=C /bin/bash --norc --noprofile"

And this is how it looks like:


I cheated, the font you see is Terminus, not Monospace. But it’s not installed by default, I highly recommend it. Check it out.

If you are reading those lines, chances are this might be useful for you. Take care!

14 October 2016 | linux | fedora

Hidden feature of Fedora 24: Live PXE boot

Some time ago, I was trying to PXE-boot Fedora 20 LiveCD directly as a squashfs filesystem from TFTP server. It was not working, because one of the dracut modules was not included on the CD/DVD init RAM disk. I filed a bug, some time passed and it’s finally fixed.

Required steps to achieve Live Fedora from PXE:

  • download Fedora 24 Live DVD (or older)
  • extract squashfs.img
  • put it on HTTP site (/var/www/htdocs)
  • extract vmlinuz and initrd.img
  • install TFTP server
  • put them into TFTP folder (subfolder boot in my case)
  • setup DHCP and PXE
  • deploy PXELinux configuration (below)
  • profit!

Example PXELinux configuration:

default menu
menu title PXE
prompt 0
timeout 200
ontimeout local

label local
  menu label ^Boot from local drive
  menu default
  localboot 0

label fedora-live
  menu label Fedora Workstation LiveBoot
  kernel boot/fedora-live/vmlinuz
  append initrd=boot/fedora-live/initrd.img root=live:http://nas.home.lan/xxx/squashfs.img ro rd.luks=0

Boot Fedora Live from your network, install it from your network when your crying friends show up with their laptops upgraded to Windows 10. Do it like a boss. End of transmission.

16 August 2016 | linux | fedora

Probing Ruby 2.0 apps with SystemTap in RHEL7

Few years ago, I wrote an article about SystemTap and Ruby in RHEL6. When RHEL 7.0 was released, things changed. It has Ruby 2.0 and the Ruby SystemTap API changed as well, therefore I am updating my old article today according to new changes.

Imagine you have a Ruby application that has some performance issues on a production server and it’s running RHEL 7.0 or newer. With SystemTap, you can easily peek into the running application and investigate bottlenecks or count memory objects. If you know DTrace from other operating systems, welcome home.

Installation of SystemTap is easy and straightforward and for our purposes we do not need to install kernel devel and debug info packages.

# yum -y install systemtap systemtap-runtime ruby

Let’s create a trivial application called factorial.rb.

# cat factorial.rb
def factorial n
  f = 1; for i in 1..n; f *= i; end; f
puts factorial(ARGV[0].to_i)

And a simple SystemTap script that shows method calls:

# cat rubycalls.stp
probe ruby.method.entry, ruby.cmethod.entry
  if (file == "factorial.rb") {
    printf("%s => %s.%s in %s:%d\n", thread_indent(1), classname, methodname, file, line);
probe ruby.method.return, ruby.cmethod.return
  if (file == "factorial.rb") {
    printf("%s <= %s.%s in %s:%d\n", thread_indent(-1), classname, methodname, file, line);

Let’s just run it for now.)

# stap rubycalls.stp -c "ruby factorial.rb 4"
 0 ruby(29131): => IO.set_encoding in factorial.rb:0
 6 ruby(29131): <= IO.set_encoding in factorial.rb:0
 0 ruby(29131): => IO.set_encoding in factorial.rb:0
 3 ruby(29131): <= IO.set_encoding in factorial.rb:0
 0 ruby(29131): => #<Class:0x00000002378c80>.core#define_method in factorial.rb:1
 6 ruby(29131):  => Module.method_added in factorial.rb:1
 9 ruby(29131):  <= Module.method_added in factorial.rb:1
45 ruby(29131): <= #<Class:0x00000002378c80>.core#define_method in factorial.rb:1
 0 ruby(29131): => String.to_i in factorial.rb:4
 2 ruby(29131): <= String.to_i in factorial.rb:4
 0 ruby(29131): => Object.factorial in factorial.rb:1
19 ruby(29131):  => Range.each in factorial.rb:2
23 ruby(29131):  <= Range.each in factorial.rb:2
25 ruby(29131): <= Object.factorial in factorial.rb:3
 0 ruby(29131): => Kernel.puts in factorial.rb:4
 7 ruby(29131):  => IO.puts in factorial.rb:4
15 ruby(29131):   => Fixnum.to_s in factorial.rb:4
18 ruby(29131):   <= Fixnum.to_s in factorial.rb:4
21 ruby(29131):   => IO.write in factorial.rb:4
40 ruby(29131):   <= IO.write in factorial.rb:4
43 ruby(29131):   => IO.write in factorial.rb:4
48 ruby(29131):   <= IO.write in factorial.rb:4
50 ruby(29131):  <= IO.puts in factorial.rb:4
52 ruby(29131): <= Kernel.puts in factorial.rb:4

Please note you have to run SystemTap under root account and also expect the first run to be a little bit slower, because SystemTap is compiling and inserting a kernel module under the hood.

SystemTap syntax is similar to C and the easiest way of learning it is reading SystemTap Beginners Guide. From the book:

SystemTap allows users to write and reuse simple scripts to deeply examine the activities of a running Linux system. These scripts can be designed to extract data, filter it, and summarize it quickly (and safely), enabling the diagnosis of complex performance (or even functional) problems.

The essential idea behind a SystemTap script is to name events, and to give them handlers. When SystemTap runs the script, SystemTap monitors for the event; once the event occurs, the Linux kernel then runs the handler as a quick sub-routine, then resumes.

There are several kinds of events; entering or exiting a function, timer expiration, session termination, etc. A handler is a series of script language statements that specify the work to be done whenever the event occurs. This work normally includes extracting data from the event context, storing them into internal variables, and printing results.

The following example will count method calls to quickly search for bottlenecks. If you don’t understand how it works, head over to the Beginners Guide for more details.

# cat rubystack.stp

global fn_calls;

probe ruby.method.entry, ruby.cmethod.entry
  fn_calls[classname, methodname] <<< 1;

probe end {
  foreach ([classname, methodname] in fn_calls- limit 30) {
    printf("%dx %s.%s\n", @count(fn_calls[classname, methodname]), classname, methodname);

  delete fn_calls;

Everytime a Ruby method is entered, counter is incremented by one in a global associative array. It prints top thirty counters on exit. When we run it, it’s a surprise!

# stap rubystack.stp -c "ruby factorial.rb 42"
2904x Module.===
1782x BasicObject.==
1782x Kernel.===
1027x Symbol.to_s
1004x Kernel.initialize_dup
1003x Kernel.dup
990x Kernel.instance_variable_set
695x String.to_s
684x Hash.[]=
660x Gem::Specification.default_value
508x String.initialize_copy
396x Array.initialize_copy
388x String.gsub
336x Kernel.class
324x Array.each
322x RbConfig.expand
320x Module.method_added
292x Array.flatten
267x File.file?
244x String.<=>
242x #<Class:0x0000000221ad70>.core#define_method
229x String.to_i
210x Enumerable.any?
185x String.strip
184x File.join
181x Regexp.=~
169x Kernel.untaint
168x Kernel.respond_to?
161x String.=~

I’d expect multiplying operation (Bignum.*) but we see equality of module instead. Since we count all the method calls, we can see what Ruby needs to done in the backround to load such a trivial example. It’s actually rubygems gem which ships with Ruby 2.0 that does the loading mechanics (and it’s poorly designed in my opinion).

Anyway, if you want to see the bignum thing, increase the parameter from 42 to let’s say 500. Now another example, slightly modified example from the SystemTap Wiki:

# cat rubytop.stp
global fn_calls
probe ruby.method.entry, ruby.cmethod.entry
  fn_calls[file, methodname, line] <<< 1
probe {
  printf("%80s %6s %30s %6s\n", "FILENAME", "LINE", "METHOD", "CALLS")
  foreach ([filename, funcname, lineno] in fn_calls- limit 15) {
    printf("%80s %6d %30s %6d\n", filename, lineno, funcname, @count(fn_calls[filename, funcname, lineno]))
probe {
  delete fn_calls

Now run the example with huge input that will cause it to loop for some time:

# stap rubytop.stp -c "ruby factorial.rb 999999999"

You should see a top-like screen which refreshes every second. The initial page will be full of method calls while from second one you will only see increasing counter of the multiply method:

                                        FILENAME   LINE                         METHOD  CALLS
                                    factorial.rb      2                              * 123483
   /usr/share/rubygems/rubygems/specification.rb   1775                            ===   3663
   /usr/share/rubygems/rubygems/specification.rb   1775                             ==   1782
   /usr/share/rubygems/rubygems/specification.rb   1779          instance_variable_set    660
   /usr/share/rubygems/rubygems/specification.rb   1779                           to_s    660
   /usr/share/rubygems/rubygems/specification.rb   1471                  default_value    660
   /usr/share/rubygems/rubygems/specification.rb   1776                            dup    594
   /usr/share/rubygems/rubygems/specification.rb   1776                 initialize_dup    594
   /usr/share/rubygems/rubygems/specification.rb   1776                initialize_copy    594
   /usr/share/rubygems/rubygems/specification.rb   1769                           to_s    330
   /usr/share/rubygems/rubygems/specification.rb   1769          instance_variable_set    330
                     /usr/lib64/ruby/rbconfig.rb    235                         expand    322
                     /usr/lib64/ruby/rbconfig.rb    236                           gsub    322
   /usr/share/rubygems/rubygems/specification.rb   1402                          file?    234
     /usr/share/rubygems/rubygems/requirement.rb     51                            ===    226

SystemTap is flexible, you can ignore some files (or directories) completely. Maybe you are only interested in code that was installed in /usr/share/project and you never want to see Kernel and Bignum classes, it’s as easy as:

probe ruby.method.entry, ruby.cmethod.entry
  if (file =~ "^/usr/share/project" && classname !~ "^(Kernel|Bignum)$") {
    fn_calls[file, methodname, line] <<< 1

By default, the counter struct resets every 5 minutes (see the second timer probe).

SystemTap Ruby markers in RHEL 7.0 offers the following probes:


It is also possible to attach to existing process:

# stap rubytop.stp -x 12345

When using Software Collections, note the correct SCL enable syntax (credit to Pavel Valena from Red Hat):

# scl enable rh-ruby22 -- stap rubystack.stp -c "ruby factorial.rb 5"

That’s all for now.

02 August 2016 | linux | fedora | systemtap

Brother MFC L27000DW and Fedora 23

I bought this new device in order to have network printing and scanning and it works just fine. It was a pain to get it working tho.

Downloaded brscan4 and brscan-skey RPM packages from Brother site and fed the DNF tool with them.

Then I visited printers built-in web interface and configured it for static IP. You can do the same on your DHCP server if you prefer to.

And then - aaarghhh - the hidden trick which caused me headache:

$ brsaneconfig4 -a name=PRACOVNA model=MFC-L2700DW ip=

And done! Switch over to “Brother *PRACOVNA” scanner and you can scan over IP. I use “simple-scan” which is quick and nice tool that aims for easy scanning into PDF with multiple pages support. Highly recommended.

To all incoming documents, payslips and letters - you will be all archived!

Actually I haven’t configured printing because I use Google Cloud Print which is build in the device, but it should not be that hard. Frankly, I tried but it did not work. Drop me a line in comments if you know how to do that.

07 July 2016 | linux | fedora