9

I wanted to store a large video file on a FAT32 drive (~18 GB), but I discovered that this isn't simply possible due to limitations of the file system.

Is there a simple tool that exists for splitting the file into smaller parts that are storable, then reassembling them when I would like to retrieve the archived file?

Or is there a better way of storing large files on FAT32?

Hennes
  • 65,142
Alex
  • 287
  • 2
    Any archive utility (eg, 7zip) can do this, with or without compression. – SLaks Sep 21 '12 at 03:05
  • Ah ok, good to know - I'll give it a shot. –  Sep 21 '12 at 03:07
  • Ok, 7zip will do it - however it takes about 5-10 times longer than the script below. Anyway, I thought it might be helpful. –  Sep 21 '12 at 03:14
  • Is this an external disk that needs to be mounted both on Windows and non-Windows systems? If not, why not just convert to a modern file system? Use whatever happens to be the default for the system that the drive is connected to. – Isaac Rabinovitch Sep 22 '12 at 02:40
  • 2
    > Ok, 7zip will do it - however it takes about 5-10 times longer than the script below.  Even without compression? o.O – Synetech Sep 22 '12 at 03:08

5 Answers5

10

Most file archivers, such as 7-Zip, WinZip, and WinRAR, allow you to split an archive across multiple files. If speed is important, you can try disabling the compression part of the program.

On GNU/Linux systems, you can use the split and cat programs from the coreutils package (e.g. split -b 4294967295 FOO /media/FATDISK/BAR to split FOO into BARaa, BARab, ... and cat /media/FATDISK/BAR?? > FOO to reassemble it). Mac OS X's command line split utility works in the same way.

PleaseStand
  • 4,919
5

If you are looking for a quick solution to this, please see the other answers featuring 7zip or split. This is more of a fun solution.

I ended up writing a small Python 2 script to achieve this.

# Author: Alex Finkel 
# Email: alex@finkel.net

This program splits a large binary file into smaller pieces, and can also


reassemble them into the original file.


To split a file, it takes the name of the file, the name of an output


directory, and a number representing the number of desired pieces.


To unsplit a file, it takes the name of a directory, and the name of an


output file.


from sys import exit, argv
from os import path, listdir


def split(file_to_split, output_directory, number_of_chunks):
    f = open(file_to_split, 'rb')
    assert path.isdir(output_directory)
    bytes_per_file = path.getsize(file_to_split)/int(number_of_chunks) + 1
    for i in range(1, int(number_of_chunks)+1):
        next_file = open(path.join(output_directory, str(i)), 'wb')
        next_file.write(f.read(bytes_per_file))
        next_file.close()
    f.close()


def unsplit(directory_name, output_name):
    assert path.isdir(directory_name)
    files = map(lambda x: str(x), sorted(map(lambda x: int(x), listdir(directory_name))))
    out = open(output_name, 'wb')
    for file in files:
        f = open(path.join(directory_name, file), 'rb')
        out.write(f.read())
        f.close()
    out.close()


if len(argv) == 4:
    split(argv[1], argv[2], argv[3])
elif len(argv) == 3:
    unsplit(argv[1], argv[2])
else:
    print "python split_large_file.py file_to_split output_directory number_of_chunks"
    print "python split_large_file.py directory name_of_output_file"
    exit()
Alex
  • 287
4

Another option: use the split command from GNU Coreutils:

split --bytes=4G infile /media/FAT32drive/outprefix

to split the file into 4 GB chunks and save the chunks to the output drive.

The original file can be recovered by concatenating the chunks (with filenames sorted alphabetically).

For usage info, see the split manual.

Coreutils, including split, should be installed by default on Linux and Mac OS X. On Windows, it's available from GnuWin32, or from Cygwin.

Mechanical snail
  • 7,693
  • great - worked with cygwin in windows - very straightforward - if you wanted to update with the recombine command that would be great, but I will look it up and suggest an edit if you don't! – tom Jan 09 '15 at 04:13
  • commented too soon - for me I needed to put --bytes=3G else it failed as 4G hit the file size limit --- but with 3G it worked a treat (well) – tom Jan 09 '15 at 04:29
3

I had exactly this problem and solved it using help from the answers already posted, but not one of them described the method exactly.

I used

split --bytes=3G infile /media/FAT32drive/outprefix

which save infile to multiple files named outprefixaa, outprefixab, outprefixac etc. with a maximum size of 3GB - when I tried to use 4GB on Fat 32 it failed.

to recombine the files I used

cat /media/FAT32drive/outprefix?? > infile

To give more specific details I did this in windows, but using a cygwin window. So the file path in my case for the external FAT32 harddisk was /cydrive/e/

tom
  • 207
0

For creating files with maximum allowed size on vfat (2³²-1 bytes), use the following command (in bash):

split --bytes=$((2**32-1)) infile /media/FAT32drive/outprefix

or, if you don't want to use bash's inline math:

split --bytes=4294967295 infile /media/FAT32drive/outprefix

The '--bytes=4G' fails, because 4G equals to 2³² bytes, which is exactly one byte more than the maximum file size on vFAT.

asdmin
  • 111