Hexdump

Hexdump is a utility that displays the contents of binary files in hexadecimal, decimal, octal, or ASCII. It's a utility for inspection, and can be use for

data recovery

, reverse engineering, and programming.

Basics

Hexdump provides output with very little effort on your part, and depending on the size of the file you're looking at, there can be a lot of output. For the purpose of this article, create a 1x1 PNG file. You can do this with a graphics application such as

GIMP

or

Mtpaint

, or you can create it in a terminal with

ImageMagick

.

Here's a command to generate a 1x1 pixel PNG with ImageMagick:

You can confirm that it's a PNG file with the `file` command:

You may wonder how the `file` command is able to determine what kind of file it is. Coincidentally, that's exactly what `hexdump` is going to reveal.

For now, you can view your 1 pixel graphic in the image viewer of your choice (spoiler: it looks like this: . ), or you can view what's inside the file with `hexdump`:

What you're seeing is the contents of the sample PNG file through a lens you may have never used before. It's the exact same data as what you see in an image viewer, encoded in a way that's unfamiliar to you.

Extracting familiar strings

Just because the default data dump seems meaningless, that doesn't mean it's devoid of valuable information. You can also translate this output, or at least the parts that actually translate, to a more familiar character set with the `--canonical` option:

In the right column, you see the same data on the left, presented as ASCII. If you look carefully, you can pick out some useful information, such as the format of the file (PNG) and even, toward the bottom, the date and time the file was created and last modified. The dots represent symbols that aren't present in the ASCII character set, which is to be expected because binary formats aren't restricted to mundane letters and numbers.

You can see that within the first 8 bytes, specifically, is the string `PNG`. That's significant because it reveals how the `file` command knows what kind of file to report. The `file` command knows from the first 8 bytes that this file is a PNG file because the

libpng specification

alerts programmers what to look for.

You can also control how many bytes hexdump displays, which is useful with files larger than 1 pixel:

You don't have to limit hexdump to PNG or graphic files. You can run hexdump against binaries you run on a daily basis, such as

ls

or

rsync

or any binary format you want to inspect.

Implementing cat with hexdump

If you read the PNG spec, you may notice that the data in the first 8 bytes looks different than what `hexdump` provides. Actually, it's the same data, but it's presented using a different conversion.

So the output of `hexdump` is true, but not always directly useful to you, depending on what you're looking for. For that reason, `hexdump` has options to format and convert the raw data it dumps.

The conversion options can get complex, so it's useful to practice with something trivial first. Here's a gentle introduction to formatting `hexdump` output by reimplementing the

cat

command with `hexdump`.

First, run `hexdump` on a text file to see its raw data. You can usually find a copy of the GPL license somewhere on your hard drive, or you can use any text file you have handy. Your output may differ, but here's how to find a copy of the GPL (or part of it):

Run `hexdump` against it:

If the output of the file you're using is very long, use the `--length` (or `-n` for short) to make it manageable for yourself.

The raw data probably means nothing to you, but you already know how to convert it to ASCII:

That's helpful, but unwieldy and difficult to read.

To format the output of `hexdump` beyond what's offered by its own options, use `--format` (or `-e`) along with some specialized formatting codes. The shorthand used for formatting is similar to what the `printf` command uses, so if you are familiar with `printf` statements, you may find `hexdump` formatting somewhat familiar.

In `hexdump`, the character sequence `%_p` tells `hexdump` to print a character in your system's default character set. All formatting notation for the `--format` option must be enclosed in *single quotes*.

The output is better, but still a little inconvenient to read because traditionally UNIX text files assume an 80 characters output width (because long ago, monitors tended to fit 80 characters across). This output isn't bound by formatting, but you can force `hexdump` to process 80 bytes at a time with a little additional formatting. Specifically, by dividing 80 by 1, you can tell `hexdump` to treat 80 bytes as 1 unit:

Now the file is being processed in 80 byte chunks, but it's lost any sense of new lines. You can add your own with the `\n` character, which on UNIX represents a new line:

You have now [approximately] implemented the `cat` command with `hexdump` formatting.

Control the output

Formatting is, realistically, how you make hexdump useful.

Now that you're familiar, in principle at least, with `hexdump` formatting, you can try to get the output of `hexdump -n 8` to match the output of the PNG header as described by the official libpng spec. First, you know that you want `hexdump` to process the PNG file in 8 byte chunks. Furthermore, you may know by integer recognition that the PNG spec is documented in decimal, which is represented by `%d` (according to the `hexdump` documentation).

You can make the output perfect by adding a blank space after each integer:

It's a perfect match with the PNG specification.

Hexdumping for fun and profit

Hexdump is a fascinating tool that not only teaches you more about how computers process and convert information, but also about how file formats and compiled binaries function. You should try running `hexdump` on files at random, throughout the day as you work. You never know what kinds of information you may find, nor when having that insight may be useful.

Proxied content from gemini://sdf.org/klaatu/geminifiles/hexdump.gmi (external content)

Gemini request details:

Original URL
gemini://sdf.org/klaatu/geminifiles/hexdump.gmi
Status code
Success
Meta
text/gemini
Proxied by
kineto

Be advised that no attempt was made to verify the remote SSL certificate.