It is often said that computers will make life a lot easier. In my opinion, that era still lies in the far future. Until then I will post here solutions, which emerge from the daily struggle with the silicon evil. Maybe they are even useful to others.



Enable HDMI and audio over HDMI on Raspberry Pi
Wednesday, 04 January 2017 19:23

This is just a note so I do not forget during the next update... If the Pi is running in a headless state most of the time and a display or projector only attached e.g. for movie viewing, Pi has to be told to enable the HDMI port and set it to a specific mode (the correct timing and resolution) at boot time.

First, boot the Pi with running display attached and query the HDMI mode using

$ tvservice -s
state 0x12000a [HDMI CEA (16) RGB lim 16:9], 1920x1080 @ 59.94Hz, progressive

This tells us that the display is running in a 1920x1080 CEA (could be DMT in your case) mode. In order to find out the correct mode number we check the available modes by running (replace CEA by DMT if this was indicated in the last command's output)

$ tvservice -m CEA
Group CEA has 16 modes:
           mode 1: 640x480 @ 60Hz 4:3, clock:25MHz progressive
           mode 2: 720x480 @ 60Hz 4:3, clock:27MHz progressive
           mode 3: 720x480 @ 60Hz 16:9, clock:27MHz progressive
           mode 4: 1280x720 @ 60Hz 16:9, clock:74MHz progressive
           mode 5: 1920x1080 @ 60Hz 16:9, clock:74MHz interlaced
           mode 6: 720x480 @ 60Hz 4:3, clock:27MHz x2 interlaced
  (prefer) mode 16: 1920x1080 @ 60Hz 16:9, clock:148MHz progressive
           mode 17: 720x576 @ 50Hz 4:3, clock:27MHz progressive
           mode 18: 720x576 @ 50Hz 16:9, clock:27MHz progressive
           mode 19: 1280x720 @ 50Hz 16:9, clock:74MHz progressive
           mode 20: 1920x1080 @ 50Hz 16:9, clock:74MHz interlaced
           mode 21: 720x576 @ 50Hz 4:3, clock:27MHz x2 interlaced
  (native) mode 31: 1920x1080 @ 50Hz 16:9, clock:148MHz progressive
           mode 32: 1920x1080 @ 24Hz 16:9, clock:74MHz progressive
           mode 33: 1920x1080 @ 25Hz 16:9, clock:74MHz progressive
           mode 34: 1920x1080 @ 30Hz 16:9, clock:74MHz progressive
 

The actual mode used in this case is number 16. Now we can edit the file /boot/config.txt to include the necessary options:

hdmi_force_hotplug=1   # make sure, the HDMI port is switched on always
hdmi_group=1           # the HDMI mode group (1: CEA, 2: DMT)
hdmi_mode=16           # the mode number we found earlier goes here

After the next reboot the display should be working at the desired resolution even when switched on later. Another issue happens when HDMI is also used for audio output. The output has to be set to HDMI as opposed to DVI. This is done by adding the line

hdmi_drive=2           # set output to HDMI and send audio over HDMI

For more details regarding the possible options, please consult the official documentation.

 
In-browser data aggregation
Sunday, 07 August 2011 07:01

Often I find myself compiling data from web pages in a form more suitable for further processing – or simply in a more accessible layout including data from sub-pages (e.g. for printing). Of course, those tasks are best done for example with some XSLT processor. However, some of these actions just require small changes to the document's DOM and are therefore easily accomplished with a line of JavaScript in the browser.

Here I want to present a small tool that can do all this by exploiting the power of XSLT in the browser within reach of a single mouse click. In this way we can take advantage of the browser's forgiving HTML-parser, which even works with documents which are not valid XML. The solution consists of a set of XSLT files, a generic script which utilises those to iteratively transform a set of pages, and a loader page which may sit on your local computer. In it all the other components are glued into a single javascript:-link. This link can then be copied to the browsers toolbar.

A tiny example could be to get an overview of all articles in this section of my homepage. In order to save bandwidth the link is not generated automatically, but please click here to create it on-the-fly from the XSLT files. This link can be used right here, or copied to the browser's toolbar to be applied on other pages as well.

It all starts with a link of the form

 
  <a class="replacer" href="/code/xslt/index.xsl,/code/xslt/list.xsl" target="text/html">
    Generate ToC
  </a>
 

For all links with the class name replacer, the required XSLT files (given in a comma-seperated list in the href-attribute) and the Javascript code of the parser are downloaded by calling generateReplacerLinks( ) from generate.js. Together this data is then written into the link's href-attribute.

Upon execution (by clicking on the final link), the function runParser from transform.js is called with four parameters:

  1. An object of the form
     
      {
        '/code/xslt/index.xsl': string1,
        '/code/xslt/list.xsl': string2
      }
     
    containing strings of XSLT documents. Those were filled in by generateReplacerLinks( ) on the basis of the file list in the href-property.
  2. The name of the first XSLT-file. It is the one to be applied to the first document.
  3. (optional) An array of MIME-types. Each one will be used to open a new browser-window to display the resulting document. If none is given, text/html is used by default. You can use this to also display the source code by specifying text/plain.
  4. (optional) The root-node where to start the transformation. It defaults to the current window's document-object.

The first XSLT-file is then applied to the current document in the current browser window and generates a new HTML document. Within those newly generated documents, any links of the form

 
  <a href="next.html" data-xsl="/code/xslt/list.xsl" data-request-type="xhr">Some link</a>
 

will be repaced by the result of applying the XSLT-file specified in the data-xsl-attribute to the document given by the href-attribute.

If the data-request-type is set to xhr, the document is obtained and parsed using an XMLHttpRequest instead of a new browser-window. This has the advantage, that additional parameters (such as the HTTP header field X-Requested-With) can be specified and no new window is required. On the down side, the document has to be strictly XML conform. In contrast to the internal HTML parser, for example non-closed tags will cause the parser to fail.

Further, parameters can be passed to the XSLT by using other data-someparam-attributes. They can be accessed by $someparam in the XSL transform after declaring them with <xsl:param name="someparam"/>. One parameter that will always be available is url, containing the URL of the current source document.

The files used in this example are:

  • The link generator generate.js which is initiated by a call to generateReplacerLinks( ).
  • The source of the transformation script used behind the scenes, transform.js.
  • The XSLT files index.xsl and list.xsl.

I usually use them on a local HTML page, calling generateReplacerLinks( ) in the onload-handler and copy the links into my toolbar for online use.

 
Firefox based presentations
Tuesday, 14 September 2010 17:59

This is just a simple example of a system that can be used to run presentations on Firefox. By utilising a combination of standards such as CSS, XBL, SVG and others, a javascript package transforms a web page into a dynamical screen presentation with lots of eye candy. By using a MozRepl the browser is able to transform formulas in TeX syntax on the fly scalable vector graphics!

If you should be interested in the content of this presentation, please have a look at the publications in the Physics section.


Presentation controls: first || previous | next || last, you're on slide number /
 
Point extraction from Google Maps
Wednesday, 30 June 2010 00:00

Sometimes, the precise coordinates of some markers from a Google Maps layer are to be obtained. One way is to look them up in the source code of the map generation. But the information should also be in the map after construction, right? For example, all the marker items on the map contain this information. The following brief script extracts the coordinates and the title of the markers and writes them into a plain text file in a new browser window.

 
  var out = '';
  var i   = 0;
  while ( x = document.getElementById( 'mtgt_unnamed_' + i ) )
    {
    out += ( [x.__marker__.C.x, x.__marker__.C.y, x.__marker__.ca.title] ).join( ', ' ) + "\n";
    i ++;
    }
  window.open( 'data:text/plain;charset=UTF-8,' + encodeURIComponent( out ) );
 

For ease of use, the whole script can be put into a bookmark. By clicking it, it will be executed on the currently shown page. So if interested, bookmark this link: GMaps point extractor.

If more information, such as content of popup windows, is needed, a script more specific to the implementation of the map generation has to be used.

 
Xournal PDF embedding wrapper
Thursday, 28 January 2010 20:58

Xournal is a great notebook application to be used on Tablet PCs. It allows for annotation of PDF documents, with the drawing/writing exported directly to the PDF. While there is vector-drawing on top, the underlying PDF is not changed (making use of PDFs updating mechanism), giving rise to a rather high quality result.

Sometimes it turns out to be inconvenient that one ends up with three files — the original PDF, the Xournal file, and the annotated PDF. Because the annotations are just stored as an update, the original PDF can always be recovered by removing the most recent update. Also, PDF allows for embedding files within the document. So in principle, a single file can be sufficient: it appears to be the fully annotated version as exported by Xournal to any PDF viewer. Additionally, the Xournal file is included which allows for editing those annotations after removing the Xournal's PDF attachments.

By making a little script out of this, annotating a PDF (and also editing an already annotated one) is as easy as

xournalpdf document.pdf

and only a single file has to be kept.

The script makes use of PDF::API2 along with PDF:API2::Names. By giving it a non-existent file name, an empty Xournal file is generated in contrast to PDF annotation. Passing an existent Xournal filename will start Xournal and embed the source in the resulting PDF.

What might be most interesting is the code to append some file attachment to a PDF: The Root-object is cloned and the name-tree of embedded files extended by an entry describing the Xournal file and also containing its encoded content.

  1. sub append( $$$ )
  2. {
  3. my ( $pdfname, $filename, $description ) = ( shift, shift, shift );
  4. my $spdf = PDF::API2::Basic::PDF::File->open( $pdfname, 1 );
  5. $description = $filename unless $description;
  6.  
  7. $spdf->{Root}->realise;
  8. $spdf->{Root} = $spdf->new_obj( $spdf->{Root} );
  9.  
  10. if ( defined $spdf->{Root}->{Names} )
  11. {
  12. $spdf->{Root}->{Names}->realise;
  13. $spdf->{Root}->{Names} = $spdf->new_obj( $spdf->{Root}->{Names} );
  14. }
  15. else
  16. { $spdf->{Root}->{Names} = PDF::API2::Basic::PDF::Dict->new; }
  17. $spdf->{Root}->{Names}->{' parent'} = $spdf;
  18. bless $spdf->{Root}->{Names}, 'PDF::API2::Names';
  19.  
  20. my $file = PDF::API2::Basic::PDF::Dict->new;
  21. @$file{'Type', ' streamfile', 'Filter'} = (
  22. PDFName( 'EmbeddedFile' ),
  23. $filename,
  24. PDFArray( PDFName( 'FlateDecode' ) )
  25. );
  26.  
  27. my $descr = PDF::API2::Basic::PDF::Dict->new;
  28. @$descr{qw/Type F EF/} = (
  29. PDFName( 'Filespec' ),
  30. PDFStr( $filename ),
  31. PDF::API2::Basic::PDF::Dict->new
  32. );
  33. $descr->{EF}->{F} = $file;
  34.  
  35. $spdf->{Root}->{Names}->set( 'EmbeddedFiles', $description, $descr );
  36. $spdf->new_obj( $descr );
  37. $spdf->new_obj( $file );
  38. $spdf->append_file;
  39. $spdf->release;
  40. }

Download: xournalpdf

 
Yet another PDF merger
Monday, 25 January 2010 22:49

Having tried all different kinds of combinations of PDF tools to simply merge files, I have been pretty disappointed. So I took the pdf-optimize.pl example from the great (but sparsely documented) PDF::API2 and wrote my own little version that has the following features:

  • Links between merged files will be converted into internal links.
  • The same is true for outlines that point to external files.
  • Trees of outlines are merged rather than just concatenated.
  • Further outlines to external files will be removed.

Links will be preserved also when using named destinations. However, other named object might become broken. For the files I encountered so far that was not a problem at all. Changing this as well should be straight-forward.

Download: pdf-merge-links.pl

 
Matlab EPS data extraction
Wednesday, 25 November 2009 11:20

Sometimes it appears to be useful to have more detailed knowledge of point data than just the plot of it – taking a ruler and measuring out coordinates of points within a diagram is simply not an option. If this plot is available in a suitable vector format such as PostScript, then it is in principle possible to read out the original data again (up to the accuracy of the plot itself).

In the case of diagrams generated with MATLAB, I can offer a little Perl-script to do this job. Appart from extracting the point sets (one CSV file is generated for each point set, PS point coordinates sit in the first two columns) it also tries to read the axes and offer a transformation from the PS coordinate system to the original one (column three and four, if available).

Keep in mind, that this is just a 15-minutes quick-and-dirty hack which works for the plots I was interested in. If you encounter any problems, write me a note. In principle, the same technique would also work for other programs such as gnuplot.

Download: matlab_extract.pl

 
Skype DBus client program for Linux
Thursday, 08 October 2009 00:08

In order to access Skype with shell scripts or Perl, I decided to use a single interface suitable for all possible uses: read/write from STDIN/STDOUT. This also allows for playing with the Skype API interactively by typing in the commands. Be careful not to enter empty lines – this may cause Skype to crash.

The program skype_dbus_client.c can be compiled with

gcc skype_dbus_client.c -o skype_dbus_client \
      `pkg-config --cflags --libs glib-2.0` \
      `pkg-config --cflags --libs dbus-glib-1`

Of course, libraries and headers of GLib and the GLib DBus bindings have to be installed.

  1. /* skype_dbus_client.c: a simple command line client to Skype's DBus interface
  2. read from STDIN and write to STDOUT */
  3.  
  4. #include <stdio.h>
  5. #include <glib.h>
  6. #include <stdlib.h>
  7. #include <string.h>
  8.  
  9. #define DBUS_API_SUBJECT_TO_CHANGE
  10. #include <dbus/dbus.h>
  11.  
  12. DBusConnection *connection;
  13.  
  14. /* iterate through DBus messages and print the string components */
  15. static void print_iter( DBusMessageIter *iter )
  16. {
  17. do
  18. {
  19. int type = dbus_message_iter_get_arg_type( iter );
  20. const char *str;
  21.  
  22. if ( type == DBUS_TYPE_INVALID ) break;
  23.  
  24. switch ( type )
  25. {
  26. case DBUS_TYPE_STRING: // it is a string message
  27. dbus_message_iter_get_basic( iter, &str );
  28. g_print( "%s\n", str );
  29. break;
  30.  
  31. case DBUS_TYPE_VARIANT: // another set of items
  32. {
  33. DBusMessageIter subiter;
  34.  
  35. dbus_message_iter_recurse( iter, &subiter );
  36. print_iter( &subiter );
  37. break;
  38. }
  39.  
  40. default:
  41. break;
  42. }
  43. }
  44. while ( dbus_message_iter_next( iter ) );
  45. }
  46.  
  47. /* the handler gets called if the DBus connection receives any data */
  48. static DBusHandlerResult notify_handler( DBusConnection *bus, DBusMessage *msg, void *user_data )
  49. {
  50. DBusMessageIter iter;
  51. dbus_message_iter_init( msg, &iter );
  52. print_iter( &iter );
  53.  
  54. return TRUE;
  55. }
  56.  
  57. /* send a string to skype */
  58. void sendToSkype( char *msg )
  59. {
  60. DBusMessageIter iter;
  61. const int reply_timeout = -1; // do not timeout -- block
  62. DBusMessage *reply;
  63. DBusMessage *message;
  64. DBusError error;
  65.  
  66. dbus_error_init( &error );
  67. message = dbus_message_new_method_call(
  68. "com.Skype.API",
  69. "/com/Skype",
  70. "com.Skype.API",
  71. "Invoke"
  72. );
  73.  
  74. if( !dbus_message_append_args( message, DBUS_TYPE_STRING, &msg, DBUS_TYPE_INVALID ) )
  75. {
  76. fprintf( stderr, "Error: reply is not except format\n" );
  77. exit( 1 );
  78. }
  79.  
  80. reply = dbus_connection_send_with_reply_and_block( connection, message, reply_timeout, &error );
  81.  
  82. if ( dbus_error_is_set( &error ) )
  83. {
  84. fprintf ( stderr, "Error: %s\n", error.message );
  85. dbus_error_free( &error );
  86. exit( 1 );
  87. }
  88.  
  89. dbus_message_iter_init( reply, &iter );
  90. print_iter( &iter );
  91.  
  92. if( dbus_error_is_set( &error ) )
  93. {
  94. fprintf( stderr, "Error: %s\n", error.message );
  95. dbus_error_free( &error );
  96. exit( 1 );
  97. }
  98. }
  99.  
  100. /* if the input is disconnected: exit the program */
  101. gboolean hangup_handler( GIOChannel *source, GIOCondition condition, gpointer data )
  102. { exit( 0 ); }
  103.  
  104. /* input waiting: read and forward to skype */
  105. gboolean input_handler( GIOChannel *source, GIOCondition condition, gpointer data )
  106. {
  107. char *b;
  108.  
  109. g_io_channel_read_line( source, &b, NULL, NULL, NULL );
  110. if ( b == NULL ) { exit( 0 ); }
  111.  
  112. sendToSkype( b );
  113. g_free( b );
  114.  
  115. return TRUE;
  116. }
  117.  
  118. int main( int argc, char **argv )
  119. {
  120. DBusObjectPathVTable vtable;
  121. GMainLoop *loop;
  122. GIOChannel *in;
  123. DBusError error;
  124.  
  125. dbus_error_init( &error );
  126. connection = dbus_bus_get( DBUS_BUS_SESSION, &error );
  127. if ( connection == NULL )
  128. {
  129. fprintf( stderr, "Failed to open connection to bus: %s\n", error.message );
  130. dbus_error_free( &error );
  131. exit( 1 );
  132. }
  133.  
  134. loop = g_main_loop_new( NULL, FALSE );
  135.  
  136. in = g_io_channel_unix_new( 0 ); // open STDIN for reading
  137. g_io_add_watch( in, G_IO_IN, input_handler, NULL ); // watch for input on STDIN
  138. g_io_add_watch( in, G_IO_HUP, hangup_handler, NULL ); // watch for hangup of STDIN
  139.  
  140. dbus_connection_setup_with_g_main( connection, NULL ); // set up te DBus connection
  141.  
  142. vtable.message_function = notify_handler; // register handler for incoming data on DBus
  143. dbus_connection_register_object_path( connection, "/com/Skype/Client", &vtable, 0 );
  144.  
  145. g_main_loop_run( loop ); // the main loop
  146.  
  147. return 0;
  148. }
 
Public access to Ordnance Survey and IGN maps
Tuesday, 17 March 2009 08:08

Ordnance Survey (the UK mapping agency) as well as IGN (the french pendant) opened their topographic maps to the public. The two services called OS OpenSpace and API Geoportail allow for integration of high-quality up-to-date topographic maps of the UK and France into private web-sites.

This integration is in both cases based on modified versions of OpenLayers which use an authenticated WMS layer (a registration key has to be used to validate the access from a specific location) as base images. Therefore it is quite easy to build special OS and GP layers to integrate these public services into standard OpenLayers projects.

For the OpenSpace layer have a look at an example of OS integration or OpenLayers sandbox.

I have implemented an IGN Geoportail layer for OpenLayers, to be downloaded here. At the moment only the maps for the french main land are included and no satellite photos. A new new layer is created by

var layer = new OpenLayers.Layer.Geoportail( "Geoportail Main Land", key );

where key contains the API key for the web page you want to access the API from. The key is transmitted on a regular basis to ensure that subsequent requests are all authenticated.

 
Toshiba's battery replacement programme
Friday, 14 November 2008 15:18

For the last years Sony stocks suffered a bit from the huge replacement calls for defect battery cells. Also Toshiba uses Sony cells for its laptop batteries. From time to time they renew the information on the website of the programme (US | Europe), appending additional models and serial numbers.

When one of my batteries (for the Portégé M400) decided not to stop loading and overheated a lot (I am glad that I was home), I discovered that this type of batteries for the M400 was actually affected as well.

Having read this on a US Toshiba site, bought from Toshiba UK and because I am residing in Germany at the moment, I had a lot of fun with three different support services on the phone:

  • The US division confirmed that these batteries should be replaced and their program identified one of them to be affected. However, shipment to Europe was not possible, this is entirely the territory of Toshiba Europe.
  • Although the european identification program did not recognise my batteries, that they are affected was confirmed by the UK support center. However – again – shipment to Germany can only be triggered by the german support.
  • In dozens of calls and emails the German support service always told me the same: The batteries are not affected and will not be replaced.
  • Finally, after quoting the statements of the other divisions in a new request, the German support center sent the batteries – without a further comment.
 
Suspend terminal sessions
Monday, 04 February 2008 18:08

Just like with X11 sessions which can be suspended by using Xvfb and xmove, suspending terminal sessions without stopping the running processes is of interest when using interactive programs where scripting is not an option.

To setup the environment all that has to be done is to run

screen

When the programs are running, the session can be put into suspend (note, that the processes continue to run and the output is buffered) by hitting ctrl-a d.

To resume the session (which could be after closing the ssh connection and re-establishing it later) simply run

screen -r
 
«StartPrev12NextEnd»

Page 1 of 2