A non-breaking space (NBSP) is a special kind of whitespace. It's an invisible signal that tells a text-processing program to avoid replacing that space with a linefeed or carriage return. For example, if you wrote "2 kg" in LibreOffice Writer and didn't want the "2" to be at…

There's always more than one way to do a job in the shell, and there may not be One Best Way to do that job, either. Nevertheless, different commands with the same output can differ in how long they take, how much memory they use and how hard they make the CPU work. Out of curiosity I trialled 6…

It's pretty easy to see which fields are empty in the tab-separated table demo (below). The 2nd and 4th field, Col2 and Col4, have no visible characters at all in the rows below the header line: That was simple, but what if the table is huge, with lots of fields and records? Eyeballing for fields…

When writing or editing scientific-type text in LibreOffice Writer, I rely a lot on its spellchecker. Unfortunately, the scientific terms I'd like to check aren't in the default dictionaries behind LibreOffice's spellchecking routine. There are two ways out of this dilemma. The first is to deal…

In my data-cleaning work I often make up tallies of selected individual characters from big, UTF-8-encoded data files. What's the best way to do this? As shown below, I've tried grep/sort/uniq, AWK and Ruby, and AWK's the fastest. The trials also revealed an unexpected problem with the uniq program…

SANITY WARNING: Please don't read this article unless you're a compulsive shell scripter, like me. Seriously. There are some excellent FOSS sequence viewers in the distro repositories, with great GUIs and lots of features. Why would you want to build a less capable viewer that only works in a…

When cleaning UTF-8 text files I sometimes come across invisible characters that I call 'gremlins'. These aren't the usual non-printing characters, like whitespace and (horizontal) tab, which are non-printing characters I expect to find in the plain text files I work with. Gremlins are weird things…

Lexical illusions are very hard to find when proofreading. The most common lexical illusion is a duplicated word, as in this well-known example: A lexical illusion: many people are not aware that the the brain will automatically ignore a second instance of the word 'the' when it starts a new line. …

I wrote the shell script described here ('grep4') to find up to four items at the same time in my data tables, which are big text files with one record per line. The script gives each searched-for item its own color on the terminal screen. It lets me know in advance how many 'hits' there are, so…