In a plain-text table, the first or header line usually contains the names of the fields. In this article I've pulled together a variety of ways to add and delete a header, and also how to do things with the rest of the table while leaving the header intact as the first line in the file. Add a…

This article compares 3 different ways to join tables on a common field. All 3 work OK on small tables, but there's a big difference in speed when the tables to be joined are large, as shown below. Table joining isn't something you need to do if you store data in a simple, flat-file database like a…

In a previous Linux Rain article I compared different ways to delete blank lines, and showed that the AWK way was the simplest and most thorough. Here I show how to split a text file into multiple text files using a surprisingly simple AWK command. Background In 2013 I published an article in Free…

In a previous Linux Rain article I explained how to use the YAD dialog to do multiple-item data entry in a GUI. Here I show one way to allow users to check and edit what they've entered in YAD. I also explain how to offer default, editable inputs in terminal-based data entry, and how to color the…

In this tutorial I explain one way to generate a random geographical location, using simple command-line tricks. Latitude and longitude: decimal degrees Latitude runs from 0 to 90 degrees from the Equator to the North Pole, and 0 to 90 degrees from the Equator to the South Pole. Longitude in…

If you just love making 'top 10'-type lists but are a little embarrassed to say so, tell people you're passionate about data exploration. To impress them even more, explain that you do your data exploration on the command line. But don't ruin the impression by telling them how easy that is! In this…

The title explains what this article is about. If you save an .odt file as text, or copy/paste the contents as a text file, or run odt2txt or the unoconv utility, you lose the apparent line structure of the original, and with it the line numbering. But there is a way... Losing apparent line…

I wrote the shell script described here ('grep4') to find up to four items at the same time in my data tables, which are big text files with one record per line. The script gives each searched-for item its own color on the terminal screen. It lets me know in advance how many 'hits' there are, so…

Big data or small data, a lot of it comes these days as plain text arranged in rows and columns. Here are a few tips on working with tables of text, both in text editors and on the command line. CSV isn't what it seems. Text tables are often exported from databases and spreadsheets as…

GNU AWK is a great tool for working with tables of data, and by using AWK arrays, you can quickly do column sums (and other calculations) based on unique values in another column (see below). Pivot tables in AWK are a bit trickier. In this article I do a slow walk through the code for a simple…