Shell scripting for software developers

In an open online edX course on Unix tools I was running over the spring with more than a thousand registered learners, I got asked for ideas on how shell scripts can be useful. This is an intriguing question, because the course focuses mainly on performing one-off tasks in the areas of software development, data engineering, and system administration, rather than automation through shell scripts. In response, I posted how shell scripting improves my personal productivity. Here’s my take on how shell scripts are employed in diverse software development tasks. I plan to post further installments on system administration and data analytics.

Continue reading "Shell scripting for software developers"

Shell scripting for personal productivity

In an edX course on Unix tools I am running these weeks, I got asked for ideas on how shell scripts can be useful. This is a very interesting question, because the course focuses mainly on performing one-off tasks in the areas of software development, data engineering, and system administration, rather than automation through shell scripts. Here’s how I’m using shell scripting to enhance my personal productivity. I’ll post further installments regarding software development and system administration.

Continue reading "Shell scripting for personal productivity"

So, you’re chairing a conference session?

Earlier today a first-time session chair asked me for tips on chairing his session. With conference presentations becoming shorter and more packed, I realized that the task of the session chair has become more demanding. Here’s some advice.

Continue reading "So, you’re chairing a conference session?"

How to avoid redoing manual corrections

Say you have an automated process to create a report, which you then have to polish by hand, because there are adjustments that require human judgment. After three hours of polishing, you realize that the report is full of errors due to a bug in the initial reporting process. Is there a way to salvage the three hours of work you put into it?

Continue reading "How to avoid redoing manual corrections"

How to Create Your Own Git Server

Although I'm a happy (also paying) user of GitHub's offerings, there are times when I prefer to host a private repository on a server I control. Setting up your own Git server can be useful if you're isolated from the public internet, if you're subject to inflexible regulations, or if you simply want features different from those offered by GitHub (and other similar providers). Setting up a Git server on a Unix (Linux, Mac OS X, *BSD, Solaris, AIX) machine isn't difficult, but there are many details to observe. Here is a complete guide.

Continue reading "How to Create Your Own Git Server"

How to Calculate an Operation's Memory Consumption

How can you determine how much memory is consumed by a specific operation of a Unix program? Valgrind's Massif subsystem could help you in this regard, but it can be difficult to isolate a specific operation from Massif's output. Here is another, simpler way.

Continue reading "How to Calculate an Operation's Memory Consumption"

Importing an RCS Project into Git

RCS stands for Revision Control System. You may have never heard it, because it's more than a quarter-century old; a paper describing it was published in 1985. Although its commands are still available in most Unix distributions and it's one of the easiest systems to use in a single-user scenario, it is clearly showing its age when compared to more modern systems. Here is how to move an existing project managed with RCS to the 21st century and Git, while preserving all its history.

Continue reading "Importing an RCS Project into Git"

Mind Mapping

In a recent NPR interview the journalist described how I used a mind map to organize my work while I served as Secretary General for Information Systems at the Greek Ministry of Finance. A number of people asked me for more details; if you're interested read on.

Continue reading "Mind Mapping"

Pretend Invitations

Choosing between people you want to invite to a function and people you have to invite is sometimes difficult. Say Alice wants to invite Tom, Dick, and Harry to a party, but she'd actually prefer if Dick didn't show up. Here's how Alice can send invitations by email from an email-capable Unix system to achieve the desired result, while covering her scheming with plausible deniability.

Continue reading "Pretend Invitations"

Synchronizing Thunderbird's Replied Flag

My main email client is Thunderbird, but I also use BlackBerry's and Apple's clients, all through the same IMAP account. Disconcertingly, email messages to which I have replied through the other clients don't show up in Thunderbird with the "replied" () icon. This means that when I browse my email using Thunderbird, I waste time trying to remember whether I have responded to a particular message. Here is how I solved the problem.

Continue reading "Synchronizing Thunderbird's Replied Flag"

Sane vim Editing of Unicode Files

Being able to use plain alphabeitc keys as editing commands is for many of us a great strength of the vi editor. It allows us to edit without hunting for the placement of the various movement keys on each particular keyboard, and, most of the time, without having to juggle in order to combine particular keys with ctrl or alt. However, this advantage can turn into a curse when editing files using a non-ASCII keyboard layout. When the keyboard input method is switched to another script (Greek in my case, or, say, Cyrillic for others) vi will stop responding to its normal commands, because it will encounter unknown characters. Here is how I've dealt with this problem.

Continue reading "Sane vim Editing of Unicode Files"

Batch Files as Shell Scripts Revisited

Four years ago I wrote about a method that could be used to have the Unix Bourne shell interpret Windows batch files. I'm using this trick a lot, because programming using the Windows/DOS batch files facilities is decidedly painful, whereas the Bourne shell remains a classy programming environment. There are still many cases where the style of Unix shell programming outshines and outperforms even modern scripting languages.

Continue reading "Batch Files as Shell Scripts Revisited"

Useful Polyglot Code

Four years ago I blogged about an incantation that would allow the Windows command interpreter (cmd) to execute Unix shell scripts written inside plain batch files. Time for an update.

Continue reading "Useful Polyglot Code"

How to Get a Glowing Recommendation Letter

Students who do well in my courses often come to me asking for a recommendation letter for graduate or postgraduate study. I only write letters for students I know well and I can honestly recommend, so some end up with a glowing recommendation while others leave empty handed. While I was drafting a few letters today, it occurred to me that obtaining a good recommendation letter is a lot easier if you've planned for it well in advance.

Continue reading "How to Get a Glowing Recommendation Letter"

Basic Etiquette of Technical Communication

Parents spend years trying to teach their children to be polite, and some of us had to learn at school how to properly address an archbishop. Yet, it seems that advice on courteousness and politeness in technical communication is in short supply; most of us learn these skills through what is euphemistically called “on the job training.” With enough bruises on my back to demonstrate the amount and variety of my experience in this area (though not my skill), here are some of the things I’ve learned.

Continue reading "Basic Etiquette of Technical Communication"

Tags for Bibliography References

I love writing my papers in LaTeX. Its declarative style allows me to concentrate on the content, rather than the form. I even format the text according to the content, keeping each phrase or logical unit on a separate line. Many publishers supply style files that format the article according to the journal's specifications. Even better, over the years I've created an extensive collection of bibliographies. I can therefore use BibTeX to cite works with a simple command, without having to re-enter their details. This also allows me to use style files to format references according to the publisher's specification. Yet, there is still the problem of navigating from a citation to the work's details. Here is how I solve it.

Continue reading "Tags for Bibliography References"

Unzipping Files in Order

Over the past couple of years I've enjoyed listening to the audio edition of the Economist newspaper. The material is superb (although I occasionally get the feeling of listening to the Voice of America), the articles are read in a clear voice, the data's encoding is plain MP3, unencumbered by digital rights (restrictions) management silliness, and the audio format is convenient to listen on the metro or while jogging. Unfortunately, the articles in the audio edition's zip file are haphazardly ordered, which, until today, marred the enjoyment of my listening.

Continue reading "Unzipping Files in Order"

In Presentations Less is More

A couple of months ago I prepared the slides for a paper I will present at the 30th International Conference on Software Engineering. After reading Garr Reynolds's book Presentation Zen: Simple Ideas on Presentation Design on Presentation Design and Delivery I became enlightened, and I decided to redo the presentation from scratch, creating less cluttered, more focused, and simpler slides.

Continue reading "In Presentations Less is More"

A Minute Minute Minder

Today I delivered the opening keynote address at the 4th Panhellenic Conference on Computer Science Education. For a number of reasons (more on that later) I wanted to keep track of my progress during the presentation. For this I put together a minute minder that displayed the time from the presentation's start and the slide I should be in. I could thus adjust my pace to finish as planned.

Continue reading "A Minute Minute Minder"

Curing MIDlet Bluetooth Disconnects

Over the last few days I've been writing a MIDlet to collect GPS coordinates and cell identifiers. I'm doing this in an effort to look at what algorithms might be needed in order to implement something similar to Google's My Location service. Here is a Google Earth example of the data I'm collecting. Yesterday, I reached a point where I was collecting all the information I needed, but the program was often plagued by random disconnections of the Bluetooth link to the GPS.

Continue reading "Curing MIDlet Bluetooth Disconnects"

Global Web Site Redirect

I recentry moved UMLGraph to its own dedicated web site. After the move a friend pointed out that all the links in Martin Fowler's web page on UmlSketchingTools would now stop working.

Continue reading "Global Web Site Redirect"

Cover Letters for Publications in LaTeX

I admit it. Although I compose most of my journal publications in LaTeX, I use Microsoft Word for writing the cover letters. The university's letterhead is provided in Word, and setting up and compiling a LaTeX document for a single text page is not worth the trouble.

Continue reading "Cover Letters for Publications in LaTeX"

A Programmer's Bookshelf

A first year student at a nearby university wrote to me asking for advice on becoming a hacker (according ESR's definition, he clarified). He sent me a laundry-list of 18 programming languages he aimed to learn by the time he graduated, and asked for other recommendations. I've learned a lot from reading books, so I compiled two reading lists for him.

Continue reading "A Programmer's Bookshelf"

The Treacherous Power of Extended Regular Expressions

I wanted to filter out lines containing the word "line" or a double quote from a 1GB file. This can be easily specified as an extended regular expression, but it turns out that I got more than I bargained for.

Continue reading "The Treacherous Power of Extended Regular Expressions"

The Power of Reusable GUI Elements

One can manipulate any graphical element of the Squeak environment by bringing up its halo: a rectangular set of icons representing actions that one can perform on any object. At first I found it cumbersome to have to go through the halo in order to perform any action, like recoloring an object or changing its name. Later I saw that this method is incredibly powerful.

Continue reading "The Power of Reusable GUI Elements"

How to Embed Citations in Diagrams

Diagrams in scientific publications occasionally link to other elements of the publication, such as bibliographic or section references. Maintaining consistency between the citations in the diagram and the publication can be tricky, but a small Perl script can automate this process.

Continue reading "How to Embed Citations in Diagrams"

The Capacitor Plague Hits a Baby Monitor

Good baby monitors are hard to come by. Many tend to be oversensitive, others switch from squelch mode to transmission with a loud hiss that is more irritating than a baby's cry. We were very satisfied with a Tomy Walkabout Digital 1998 baby monitor, until the day it started emitting a squeaking sound.

Continue reading "The Capacitor Plague Hits a Baby Monitor"

So Long as there's a Jingle in your Head, Television isn't Free

Yesterday I switched from an ancient version of the "free" Adobe Reader to the current version 7.0. I spent the morning studying some fairly tricky technical documents. Within that interval I often caught my eyes glancing to the top right of the Adobe Reader's display window where an advert button flashed as it changed its content. Needless to say, this change of focus interrupted my train of thought, and got me out of "flow mode".

Continue reading "So Long as there's a Jingle in your Head, Television isn't Free"

White Noise Calms Babies

A week ago I told my colleague Damianos Chatziantoniou that drying the body of a newborn baby with a hair drier keeps it consistently dry avoiding rashes, and also calms the baby down. Today he told me the advice worked wonders. Many parents have discovered that sources of white noise, like the sound of a vacuum cleaner or a hair drier, seem to calm down a baby's crying spell.

Continue reading "White Noise Calms Babies"

UML Class Diagrams from C++ Code

I needed a UML class diagram of the classes I use in the implementation of CScout refactoring browser. I drew the last such diagram on paper about four years ago, so it was definitely out of date. I always say that whenever possible documentation should be automatically generated from the code, so I decided to automate the task.

Continue reading "UML Class Diagrams from C++ Code"

Efficient Human Multitasking

I sometimes hear colleagues complaining that they can't get anything done, because they have too many tasks in their head. I've found that in order to increase the efficiency of my work I need a moderately large selection of pending tasks. This allows me to match the type of work I can do at a given moment with a task in the most optimal way.

Continue reading "Efficient Human Multitasking"

Batch Files as Shell Scripts

Although the Unix Bourne shell offers a superb environment for combining existing commands into sophisticated programs, using a Unix shell as an interactive command environment under Windows can be painful.

Continue reading "Batch Files as Shell Scripts"

Debuggers and Logging Frameworks

As soon as we started programming, we found to our surprise that it wasn't as easy to get programs right as we had thought. Debugging had to be discovered.

— Maurice Wilkes discovers debugging, 1949

Continue reading "Debuggers and Logging Frameworks"

Public Bookmarking

You're searching the internet to answer a question you have, and after some painstaking detective work you locate the answer. Where do you store the answer for future reference?

Continue reading "Public Bookmarking"

Bug Busters

Although only a few may originate a policy, we are all able to judge it.

— Pericles of Athens

Continue reading "Bug Busters"

How to Sort Three Numbers

Quick: how do you sort three numbers in ascending order?

Continue reading "How to Sort Three Numbers"

Supporting Java's Foreach Construct

Java 1.5 supports a new foreach construct for iterating over collections. The construct can be used on arrays and on all classes in Java's Collection framework. I searched the internet for an example on how to make my own classes iterable with this construct, but could not find an example.

Continue reading "Supporting Java's Foreach Construct"

Hard Disk Failure

I tell everybody that the question is not whether your hard drive will fail, but when it will fail. My laptop's drive started emmitting a loud grinding sound last Saturday.

Continue reading "Hard Disk Failure"

Dear Editor

Machines should work. People should think.

— Richard Hamming

Continue reading "Dear Editor"

Macro-based Substitutions in Source Code

A friends asks: "How can one easily replace a method call (which can contain arguments with brackets in its invocation code) with a simple field access?

Continue reading "Macro-based Substitutions in Source Code"

Who Maintains the GNU Plotutils?

The GNU plotutils package contains a reimplementation of a pic language processor. I find pic invaluable for drawing diagrams; in fact the sequence diagram editor in my UMLGraph system, depends on it. The current version is 2.4.1, and was released in July 2000 - almost four years ago. I have discovered two bugs, but no one seems to be maintaining the package. This is unfortunate.

Continue reading "Who Maintains the GNU Plotutils?"

Continous Bookmarking

When editing documents or code, my not so agile fingers, often trigger a movement or search command that accidentally throws me to a random location in the text I am editing. How can I return back? Amazingly, I noticed I am using exactly the same trick for returning back on both the vim editor I use for most of my editing tasks, and Microsoft Word I use for collaborating with many colleagues.

Continue reading "Continous Bookmarking"

Patching Framework III

Time warp. I needed to read some old files I wrote in 1992 using the Ashton-Tate Framework III program. Unfortunately, trying to run the program under Windows XP resulted in a "Divide overflow" error. A bit of searching on the web revealed that the problem was related to the system's speed (1.6GHz). Apparently, Framework tries to calculate the speed of the machine by dividing a fixed number with a loop counter; on modern machines this results in the overflow.

Continue reading "Patching Framework III"