It is often said that computers will make life a lot easier. In my opinion, that era still lies in the far future. Until then I will post here solutions, which emerge from the daily struggle with the silicon evil. Maybe they are even useful to others.
In-browser data aggregation
Sunday, 07 August 2011 07:01
Often I find myself compiling data from web pages in a form more suitable for further processing – or simply in a more accessible layout including data from sub-pages (e.g. for printing). Of course, those tasks are best done for example with some XSLT processor. However, some of these actions just require small changes to the document's DOM and are therefore easily accomplished with a line of JavaScript in the browser.
Here I want to present a small tool that can do all this by exploiting the power of XSLT in the browser within reach of a single mouse click. In this way we can take advantage of the browser's forgiving HTML-parser, which even works with documents which are not valid XML. The solution consists of a set of XSLT files, a generic script which utilises those to iteratively transform a set of pages, and a loader page which may sit on your local computer. In it all the other components are glued into a single javascript:-link. This link can then be copied to the browsers toolbar.
A tiny example could be to get an overview of all articles in this section of my homepage. In order to save bandwidth the link is not generated automatically, but please click here to create it on-the-fly from the XSLT files. This link can be used right here, or copied to the browser's toolbar to be applied on other pages as well.
For all links with the class name replacer, the required XSLT files (given in a comma-seperated list in the href-attribute) and the Javascript code of the parser are downloaded by calling generateReplacerLinks( ) from generate.js. Together this data is then written into the link's href-attribute.
Upon execution (by clicking on the final link), the function runParser from transform.js is called with four parameters:
containing strings of XSLT documents. Those were filled in by generateReplacerLinks( ) on the basis of the file list in the href-property.
The name of the first XSLT-file. It is the one to be applied to the first document.
(optional) An array of MIME-types. Each one will be used to open a new browser-window to display the resulting document. If none is given, text/html is used by default. You can use this to also display the source code by specifying text/plain.
(optional) The root-node where to start the transformation. It defaults to the current window's document-object.
The first XSLT-file is then applied to the current document in the current browser window and generates a new HTML document. Within those newly generated documents, any links of the form
will be repaced by the result of applying the XSLT-file specified in the data-xsl-attribute to the document given by the href-attribute.
If the data-request-type is set to xhr, the document is obtained and parsed using an XMLHttpRequest instead of a new browser-window. This has the advantage, that additional parameters (such as the HTTP header field X-Requested-With) can be specified and no new window is required. On the down side, the document has to be strictly XML conform. In contrast to the internal HTML parser, for example non-closed tags will cause the parser to fail.
Further, parameters can be passed to the XSLT by using other data-someparam-attributes. They can be accessed by $someparam in the XSL transform after declaring them with <xsl:param name="someparam"/>. One parameter that will always be available is url, containing the URL of the current source document.
The files used in this example are:
The link generator generate.js which is initiated by a call to generateReplacerLinks( ).
The source of the transformation script used behind the scenes, transform.js.
I usually use them on a local HTML page, calling generateReplacerLinks( ) in the onload-handler and copy the links into my toolbar for online use.
Firefox based presentations
Tuesday, 14 September 2010 17:59
This is just a simple example of a system that can be used to run presentations on Firefox. By utilising a combination of standards such as CSS, XBL, SVG and others, a javascript package transforms a web page into a dynamical screen presentation with lots of eye candy. By using a MozRepl the browser is able to transform formulas in TeX syntax on the fly scalable vector graphics!
If you should be interested in the content of this presentation, please have a look at the publications in the Physics section.
Sometimes, the precise coordinates of some markers from a Google Maps layer are to be obtained. One way is to
look them up in the source code of the map generation. But the information should also be in the map after construction, right? For example, all the marker items on the map contain this information. The following brief script extracts the coordinates and the title of the markers and writes them into a plain text file in a new browser window.
var out = '';
var i = 0;
while( x = document.getElementById('mtgt_unnamed_' + i )){
out += ([x.__marker__.C.x, x.__marker__.C.y, x.__marker__.ca.title]).join(', ') + "\n";
i ++;
}
window.open('data:text/plain;charset=UTF-8,' + encodeURIComponent( out ));
For ease of use, the whole script can be put into a bookmark. By clicking it, it will be executed on the currently shown page. So if interested, bookmark this link: GMaps point extractor.
If more information, such as content of popup windows, is needed, a script more specific to the implementation of the map generation has to be used.
Xournal PDF embedding wrapper
Thursday, 28 January 2010 20:58
Xournal is a great notebook application to be used on Tablet PCs.
It allows for annotation of PDF documents, with the drawing/writing exported directly to the PDF. While there is
vector-drawing on top, the underlying PDF is not changed (making use of PDFs updating mechanism), giving rise to a rather high quality result.
Sometimes it turns out to be inconvenient that one ends up with three files — the original PDF, the Xournal file, and the annotated PDF. Because the annotations are just stored as an update, the original PDF can always be recovered by removing the most recent update. Also, PDF allows for embedding files within the document. So in principle, a single file can be sufficient: it appears to be the fully annotated version as exported by Xournal to any PDF viewer. Additionally, the Xournal file is included which allows for editing those annotations after removing the Xournal's PDF attachments.
By making a little script out of this, annotating a PDF (and also editing an already annotated one) is as easy as
xournalpdf document.pdf
and only a single file has to be kept.
The script makes use of PDF::API2 along with PDF:API2::Names. By giving it a non-existent file name, an empty Xournal file is generated in contrast to PDF annotation. Passing an existent Xournal filename will start Xournal and embed the source in the resulting PDF.
What might be most interesting is the code to append some file attachment to a PDF: The Root-object is cloned and the name-tree of embedded files extended by an entry describing the Xournal file and also containing its encoded content.
Having tried all different kinds of combinations of PDF tools to simply merge files, I have been
pretty disappointed. So I took the pdf-optimize.pl example from the great (but sparsely documented) PDF::API2 and wrote my own little version that has the following features:
Links between merged files will be converted into internal links.
The same is true for outlines that point to external files.
Trees of outlines are merged rather than just concatenated.
Further outlines to external files will be removed.
Links will be preserved also when using named destinations. However, other named object might become broken.
For the files I encountered so far that was not a problem at all. Changing this as well should be straight-forward.
Sometimes it appears to be useful to have more detailed knowledge of point data than just the plot of it –
taking a ruler and measuring out coordinates of points within a diagram is simply not an option. If this plot is available in a suitable vector format such as PostScript, then it is in principle possible to read out the original data again (up to the accuracy of the plot itself).
In the case of diagrams generated with MATLAB, I can offer a little Perl-script to do this job. Appart from extracting the point sets (one CSV file is generated for each point set, PS point coordinates sit in the first two columns) it also tries to read the axes and offer a transformation from the PS coordinate system to the original one (column three and four, if available).
Keep in mind, that this is just a 15-minutes quick-and-dirty hack which works for the plots I was interested in. If you encounter any problems, write me a note. In principle, the same technique would also work for other programs such as gnuplot.
In order to access Skype with shell scripts or Perl, I decided to use a single interface suitable
for all possible uses: read/write from STDIN/STDOUT. This also allows for playing with the Skype API interactively
by typing in the commands. Be careful not to enter empty lines – this may cause Skype to crash.
Ordnance Survey (the UK mapping agency) as well as IGN (the french pendant) opened their topographic maps to the public. The two services called OS OpenSpace and API Geoportail allow for integration of high-quality up-to-date topographic maps of the UK and France into private web-sites.
This integration is in both cases based on modified versions of OpenLayers which use an authenticated WMS layer (a registration key has to be used to validate the access from a specific location) as base images. Therefore it is quite easy to build special OS and GP layers to integrate these public services into standard OpenLayers projects.
I have implemented an IGN Geoportail layer for OpenLayers, to be downloaded here. At the moment only the maps for the french main land are included and no satellite photos. A new new layer is created by
var layer = new OpenLayers.Layer.Geoportail("Geoportail Main Land", key );
where key contains the API key for the web page you want to access the API from.
The key is transmitted on a regular basis to ensure that subsequent requests are all authenticated.
Toshiba's battery replacement programme
Friday, 14 November 2008 15:18
For the last years Sony stocks suffered a bit from the huge replacement calls for defect battery cells. Also Toshiba uses Sony cells for its laptop batteries. From time to time they renew the information on the website of the programme (US | Europe), appending additional models and serial numbers.
When one of my batteries (for the Portégé M400) decided not to stop loading and overheated a lot (I am glad that I was home), I discovered that this type of batteries for the M400 was actually affected as well.
Having read this on a US Toshiba site, bought from Toshiba UK and because I am residing in Germany at the moment, I had a lot of fun with three different support services on the phone:
The US division confirmed that these batteries should be replaced and their program identified one of them to be affected. However, shipment to Europe was not possible, this is entirely the territory of Toshiba Europe.
Although the european identification program did not recognise my batteries, that they are affected was confirmed by the UK support center. However – again – shipment to Germany can only be triggered by the german support.
In dozens of calls and emails the German support service always told me the same: The batteries are not affected and will not be replaced.
Finally, after quoting the statements of the other divisions in a new request, the German support center sent the batteries – without a further comment.
Suspend terminal sessions
Monday, 04 February 2008 18:08
Just like with X11 sessions which can be suspended by using Xvfb and xmove, suspending terminal sessions without stopping the running processes is of interest when using interactive programs where scripting is not an option.
To setup the environment all that has to be done is to run
screen
When the programs are running, the session can be put into suspend (note, that the processes continue to run and the output is buffered) by hitting ctrl-a d.
To resume the session (which could be after closing the ssh connection and re-establishing it later) simply run
screen -r
X11 Remote Control
Monday, 28 January 2008 15:36
x2x is a brilliant little remote control for X11. For example when you are working at different computers simultaneously it can make life much easier. Mouse and keyboard commands are sent to other xservers as well as it copies the clipboard. The standard X11 protocol is used, so the usual tricks and ssh tunnels work fine.
I have a shortcut on my laptop to “invite” the keyboard and mouse of my desktop at work. To avoid exposing the xserver, a bit of ssh tunneling is done:
ssh -f -n -X desktop.at.work
'export DISPLAY=:0.0;ssh -n -f -X cap@`echo $SSH_CLIENT | cut -d" " -f1` x2x -west -to :1.0';
echo"connecting X server ..." |
osd_cat -A right -s 2 -f -*-helvetica-bold-r-*-*-20-*-*-*-*-*-*-*
After that, the mouse pointer can leave the desktop screen on the left (west) and will enter the laptop screen on the right. Whichever screen the mouse is residing in will receive the keyboard events from the desktop keyboard. osd_cat is merely used to print a notification of what is happening on the display.