Shell scripting for personal productivity
In an edX course on Unix tools I am running these weeks,
I got
asked for ideas on how shell scripts can be useful.
This is a very interesting question, because the course focuses mainly
on performing one-off tasks in the areas of software development,
data engineering, and system administration, rather than automation
through shell scripts.
Here’s how I’m using shell scripting to enhance my personal productivity.
I’ll post further installments regarding software development
and system administration.
Continue reading "Shell scripting for personal productivity"Last modified: Monday, March 23, 2020 11:43 am
Modular SQL Queries with Unit Tests
I’m sure I’m not the only person on earth facing a
complex and expensive analytical processing task.
The one I’ve been working on for the past couple of years,
runs on the GHTorrent 98.5 GB data set of
GitHub process data.
It comprises 99 SQL queries (2599 lines of SQL code in total)
and takes more than 20 hours to run on a hefty server.
To make the job’s parts run efficiently and reliably I implemented
simple-rolap,
a bare-bones relational online analytical processing tool suite.
To ensure the queries produce correct results,
I wrote RDBUnit,
a unit testing framework for relational database queries.
Here is a quick overview on how to use the two.
Continue reading "Modular SQL Queries with Unit Tests"Last modified: Sunday, August 5, 2018 2:01 pm
Unix Architecture Evolution Diagrams
Today I put online two
diagrams depicting the architecture of the Unix operating system,
one for the 1972 First Research Edition and one for FreeBSD,
one of its direct descendants.
Here are the details on how I created these diagrams.
Continue reading "Unix Architecture Evolution Diagrams"Last modified: Wednesday, May 10, 2017 4:20 pm
Chemical Element Trump Cards
I can still remember the weight of the Lancia Stratos car that featured
in the Top Trumps
cards were playing as children in the 1970s: 870kg.
It was the lightest of all the flashy cars in the set,
and therefore a much sought-after card.
Other card sets that kept us busy included airplanes, motorcycles, and tanks.
Through them we learned tens of useless trivia,
but also got a feeling of the compromises inherent in engineering.
Wondering whether I could leverage such a game to make it even more educational,
I created a set of trump cards containing properties of chemical elements.
Continue reading "Chemical Element Trump Cards"Last modified: Friday, January 11, 2013 9:02 am
How to Decrypt "Secrets for Android" Files
Secrets for Android
is a nifty Android application that allows you to securely store
passwords and other sensitive data on your Android phone.
Your data are encoded with your supplied password using strong
cryptography and are therefore protected if your phone gets stolen.
Although the application offers a backup and an export facility,
I found both wanting in terms of the availability and confidentiality
associated with their use.
Continue reading "How to Decrypt "Secrets for Android" Files"Last modified: Thursday, February 9, 2012 0:19 am
Synchronizing Thunderbird's Replied Flag
My main email client is Thunderbird, but I also use BlackBerry's and Apple's
clients,
all through the same IMAP account.
Disconcertingly, email messages to which I have
replied through the other clients don't show up in Thunderbird with the
"replied" (
) icon.
This means that when I browse my email using Thunderbird,
I waste time trying to remember whether I have responded
to a particular message.
Here is how I solved the problem.
Continue reading "Synchronizing Thunderbird's Replied Flag"Last modified: Thursday, October 14, 2010 1:40 am
Email Analytics
During the past six months I've been drowning in email.
I spend a large part of my day responding to email messages and filing
incoming messages I consider important.
Yet I'm falling behind
and this affects the quality of my work:
I sometimes delay responding to important messages.
Followng Peter Drucker's dictum
"If you can't measure it, you can't manage it",
I decided to write a tool to analyze my incoming and outgoing
email messages.
Continue reading "Email Analytics"Last modified: Thursday, May 20, 2010 12:40 am
HP-200LX Remote Control Hacks
All my friends know that for the past 15 years I've been semi-attached
to an
HP 100LX
palmtop PC
(recently updated to a 200LX)
for my personal information management and many other tasks.
The device is extremely versatile, sturdy, and flexible.
Amazingly, after so many years of hard daily use, I still find new
applications for it.
Continue reading "HP-200LX Remote Control Hacks"Last modified: Friday, August 21, 2009 0:27 am
Real-Time Google Earth GPS Tracking
In a recent trip I incorrectly assumed that real-time tracking of
Google Earth's pre-cached maps with a GPS receiver would be sufficient
help for navigating around the highways in Los Angeles.
I therefore experimented with the way Google Earth's
sparsely-documented real time tracking works,
and wrote a small program to interface Google Earth with a GPS receiver.
Fortunately, after seeing a colleague drive with a car-GPS device
on the dashboard I came to my senses, and got a real
Garmin Nuvi
car-GPS device.
Continue reading "Real-Time Google Earth GPS Tracking"Last modified: Wednesday, July 1, 2009 4:42 pm
Greek Numerals in OpenOffice.org
OpenOffice.org
doesn't support Greek numerals,
and this is a problem for its Greek localization,
because such numerals are often used for section and list numbering.
As an exercise in large scale
code reading and in the
writing of code I'm supposed to teach to undergraduate students,
I decided to contribute an implementation to OpenOffice.org.
Continue reading "Greek Numerals in OpenOffice.org"Last modified: Thursday, June 25, 2009 9:27 pm
Revisiting the Antikythera Mechanism Emulator
Over the past few weeks I updated the
Antikythera mechanism emulator
I built in 2007.
I was preparing for an invited talk on the subject, which I'll give at the
2009 USENIX Annual Technical Conference,
and for this I wanted to include in the emulator the new findings
recently published in Nature.
Continue reading "Revisiting the Antikythera Mechanism Emulator"Last modified: Wednesday, May 27, 2009 12:10 am
Fixing the Orientation of JPEG Photographs
I used to fix the orientation of my photographs through an application
that would transpose the compressed JPEG blocks.
This had the advantage of avoiding the image degradation of a
decompression and a subsequent compression.
Continue reading "Fixing the Orientation of JPEG Photographs"Last modified: Sunday, June 14, 2009 8:20 pm
A Tiling Demo
Over the past (too many) days I've been preparing my presentation for the
ACCU 2009
conference.
At one point I wanted to show how loop tiling increases locality of reference
and therefore cache hits.
Surprisingly, I could not find a demo on the web, so I built one from scratch.
Here are two applets demonstrating memory accesses during a matrix raise to the
power of two operation.
Continue reading "A Tiling Demo"Last modified: Tuesday, April 21, 2009 5:39 pm
Monitor Process Progress on Unix
I often run file-processing commands that take many hours to
finish, and I therefore need a way to monitor their progress.
The Perkin-Elmer/Concurrent OS32 system I worked-on for a couple
of years back in 1993 (don't ask)
had a facility that displayed for any executing
command the percentage of work that was completed.
When I first saw this facility working on the programs I maintained,
I couldn't believe my eyes, because I was sure that those rusty
Cobol programs didn't contain any functionality to monitor their progress.
Continue reading "Monitor Process Progress on Unix"Last modified: Monday, October 27, 2008 1:34 pm
Suspend Windows from the Command Line
I used to leave my computer up all night, but I've come to realize that this
is ecologically unsound.
Now I suspend it before going to sleep, but this missed running
a daily job that used to run at 03:00 am.
The job marks my students' exercises and send me email with the next day's
appointments.
I thus decided to schedule the task to wakeup my computer at 3:00 am,
run the job, and then suspend it again.
The Windows scheduler allows you to specify a wakeup option,
but not a subsequent suspend.
Furthermore, it seems that Windows lacks a way to suspend from the
command line (while maintaining the ability to hibernate), and the
only free tools on the web are distributed in executable form,
so I ended writing a small tool myself.
Continue reading "Suspend Windows from the Command Line"Last modified: Monday, October 6, 2008 7:25 pm
Saving the Editor's History
I recently spent a few days writing some tricky bit-twiddling code to
implement a radix tree.
I found myself making many programming mistakes, and I thought it would be
interesting to study them, examine their contributing factors, and
think how each of them could be prevented.
Continue reading "Saving the Editor's History"Last modified: Monday, August 25, 2008 4:32 pm
A Minute Minute Minder
Today I delivered the opening
keynote address
at the 4th Panhellenic Conference on Computer Science Education.
For a number of reasons (more on that later) I wanted to keep track of
my progress during the presentation.
For this I put together a minute minder that displayed the
time from the presentation's start and the slide I should be in.
I could thus adjust my pace to finish as planned.
Continue reading "A Minute Minute Minder"Last modified: Saturday, March 29, 2008 6:22 am
How to Embed Citations in Diagrams
Diagrams in scientific publications occasionally link to other elements
of the publication, such as bibliographic or section references.
Maintaining consistency between the citations in the diagram
and the publication can be tricky, but a small Perl script
can automate this process.
Continue reading "How to Embed Citations in Diagrams"Last modified: Thursday, October 13, 2016 3:40 pm
White Noise Calms Babies
A week ago I told my colleague
Damianos Chatziantoniou
that drying the body of a newborn baby with a hair drier keeps
it consistently dry avoiding rashes, and also calms the baby down.
Today he told me the advice worked wonders.
Many parents have discovered that sources of white noise, like the sound
of a vacuum cleaner or a hair drier, seem to calm down a baby's
crying spell.
Continue reading "White Noise Calms Babies"Last modified: Wednesday, November 15, 2006 7:54 am
Converting RIS to BibTeX
Digital libraries increasingly provide an option to export bibliographic
data.
Unfortunately, many, like
IEEE Xplore,
SpringerLink, and
Scopus
don't support the BibTeX format I use for storing
my bibliographies.
(To its credit the
ACM Portal offers a BibTeX
export option.
On the other hand, Elsevier's
ScienceDirect and
JSTOR don't offer
any export facility.)
Continue reading "Converting RIS to BibTeX"Last modified: Wednesday, November 8, 2006 10:01 am
Patching Framework III
Time warp.
I needed to read some old files I wrote in 1992 using the Ashton-Tate
Framework III program.
Unfortunately, trying to run the program under Windows XP resulted in a
"Divide overflow" error.
A bit of searching on the web revealed that the problem was related
to the system's speed (1.6GHz).
Apparently, Framework tries to calculate the speed of the machine
by dividing a fixed number with a loop counter;
on modern machines this results in the overflow.
Continue reading "Patching Framework III"Last modified: Thursday, August 12, 2004 9:47 am
Binary File Similarity Checking
How can one determine whether two binary files
(for example, executable images) are somehow similar?
I started writing a program to perform this task.
Such a program could be useful for determing
whether a vendor had included GNU
Public License (GPL)
code in a propriatary product, violating the GPL license.
After writing about 20 lines, I realized that I needed an accurate
definition of similarity than the vague
"the two files contain a number of identical subsequences"
I had in mind.
Continue reading "Binary File Similarity Checking"Last modified: Friday, March 19, 2004 2:15 pm
