split-overlap.py is a tool to split a binary file in parts of a given size.

For example: split-overlap.py 1000 test.data

When test.data is a binary file with size 2500 bytes, the above command will create 2 files of 1000 bytes and one file of 500 bytes.

It’s also possible to split a file with some overlap. Like this:

The blue block represents the original file, the yellow blocks are parts of the original file without overlap, and the green blocks represent parts of the original file with some overlap.

A command to achieve this, is, for example: split-overlap.py 100M+1M dump

This will create parts of 101 MB in size, with a overlap of 1 MB.

The main reason I developed this tool, is to be able to handle very large files, like memory dumps, by tools who can not handle such large files.

Splitting up a file in smaller, equal parts is a solution, but then you run the risk (a small risk) that the pattern you are looking for, is just at the “edge”: that the file is split in such a way, that one part contains the beginning of the pattern, and the next part contains the rest of the pattern. Then your tools are unlikely to find the pattern.

I solve this with my tool by using an overlap. You just have to make sure that the size of the overlap, is larger than the pattern you are looking for.

If you want to know more, read the man page: split-overlap.py -m