Repurpose your gig of Gmail as a networked filesystem .
I wouldn't give for a spare gig of networked filesystem on which to stash a backup of my work in progress or as an intermediary between two firewalled systems (thus not directly reachable from one to the other).
GmailFS (http://richard.jones.name/google-hacks/gmail-filesystem/gmail-filesysteml) puts your gigabyte of Gmail storage to work for just such a purpose. It provides a mountable Linux filesystem repurposing your Gmail account as its storage medium.
GmailFS is a Python application that uses the
FUSE (http://sourceforge.net/projects/avf) userland
filesystem infrastructure to help provide a filesystem and the
libgmail ( Hack #80
GmailFS supports most file operations, such as read, write, open, close, stat, symlink, link, unlink, truncate, and rename. This means that you can use the lion's share of your favorite Unix command-line tools (cp, ls, mv, rm, ln, grep, et al) to operate on files stored on Google's Gmail servers.
So, what might you store on and do with the Gmail filesystem? About anything that you would with any other (possibly unreliable) networked filesystem built on a cool hack or three. Figure 6-28 shows the Firefox web browser launched from an executable stored as a message in my Gmail account.
All meta-information in the GmailFS is stored in the subjects of emails sent by the Gmail user to themselves.
|
The actual file data is stored in attachments. Files can span several attachments, allowing file sizes greater than the maximum Gmail attachment. File size should be limited only by the amount of free space in your Gmail account.
There are three types of important structures in the GmailFS:
Directory and file entry structures hold the parent path and name of files or directories. Symlink information is also kept here. These structures have a reference to the file's or directory's inode (a data structure holding information about where and how the file or directory is stored).
Inode structures hold the kind of information usually found in a Unix inode, such as mode, uid, gid, size, etc.
Data block structures are one of three types of messages GmailFS uses to store information related to the filesystem. The subject of the messages holding these structures contains a reference to the file's inode as well as the current block number.
As GmailFS can store files longer than the maximum Gmail attachment size, it uses block numbers to refer to the slice of the original file that this data block message refers to. For example, if you have a blocksize of 5 MB and a file 22 MB long, you will have five blocks (5 MB, 5 MB, 5 MB, 5 MB, and 2 MB); the block numbers for these will be 0, 1, 2, 3, and 4, respectively.
All subject lines contain an
fsname (filesystem name) field that serves two purposes.
Prevents the injection of spurious data into the filesystem by external attackers. As such, the fsname should be chosen with the same care that you would exercise in choosing a password.
Allows multiple filesystems to be stored on a single Gmail account. By mounting with different fsname options set, the user can create distinct filesystems.
This isn't for the uninitiated. I haven't provided newbie-focused step-by-step installation instructions, because if you aren't able to take care of some of these details yourself, you probably shouldn't be mucking about in this hack. If you're out of your depth, sit back, relax, and read on for edification's sake.
Before you begin, make sure you have Python 2.3 and the python2.3-dev packages installed.
Install Version 1.3 of FUSE (http://sourceforge.net/projects/avf). Some Linux distributions (such as Debian) make this available as a package. If your distro doesn't, you'll need to download the source (http://sourceforge.net/project/showfiles.php?group_id=21636) and make and install it manually.
Next you'll need the Python FUSE bindings (http://richard.jones.name/google-hacks/gmail-filesystem/fuse-python.tar.gz). Download and extract fuse-python.tar.gz and follow the instructions in fuse-python/INSTALL.
|
Grab libgmail (http://sourceforge.net/project/showfiles.php?group_id=113492)
Hack #80
Finally, download GmailFS (http://richard.jones.name/google-hacks/gmail-filesystem/gmailfs.tar.gz) itself and unarchive it. Copy gmailfs.py to somewhere easily accessible (/usr/local/bin/gmailfs.py, for example) and mount.gmailfs (a modified version of mount.fuse distributed with FUSE 1.3) to /sbin/mount.gmailfs.
|
Take a moment to enjoy just how much you know about such things and move on when you're ready.
All that remains is to mount your Gmail filesystem.
You can do so via fstab or on the command line using mount. To use fstab, create an /etc/fstab entry that looks something like this:
/usr/local/bin/gmailfs.py/path/of/mount/point gmailfs \ noauto,username=
gmailuser ,password=
gmailpass ,fsname=
zOlRRa
Replace gmailuser and gmailpass with your Gmail username and password, respectively. The value you pass to fsname is one you'd like to dub this Gmail filesystem.
|
To mount the filesystem from the command line, use the following command:
# mount -t gmailfs /usr/local/bin/gmailfs.py/path/of/mount/point \ -o username=
gmailuser ,password=
gmailpass ,fsname=
zOlRRa
Again, replace gmailuser, gmailpass, and zOlRRa with your Gmail username, Gmail password, and preferred filesystem name.
|
Figure 6-28 shows my mounted gmailfs filesystem in action.
There are a few things you should know as you start strolling about and storing things on your Gmail filesystem:
GmailFS also has a blocksize option, the default being 5 MB. Files smaller than the minimum blocksize will only use the amount of space required to store the file,
not the full blocksize . Note that any files created during a previous mount with a different blocksize will retain their original blocksize until deleted. For most applications you will make best use of your bandwidth by keeping the blocksize as large as possible.
When you delete files, GmailFS will place the files in the trash. The libgmail library does not currently support purging items from the trash, so you will have to do this manually through the regular Gmail web interface.
To avoid seeing the messages created for your Gmail filesystem in your inbox, you probably want to create a filter (http://gmail.google.com/support/bin/answer.py?answer=6579&query=filter&topic=&type=f) to automatically archive GmailFS messages as they arrive in your inbox. The best approach is probably to search for the fsname value; it'll be in the subject of all your GmailFS messages.
At the time of this writing, there are some outstanding issues with GmailFS that you should be aware of:
I don't recommend storing your only copy of anything important on GmailFS for the following two reasons:
GmailFS is currently a 0.2 release and should be treated as such. You can depend on its being undependable.
There's no cryptography involved, so your files will all be stored in plain text on Google's Gmail servers. This will no doubt make some of you nervous.
Performance is acceptable for uploading and downloading very large files (obviously dependent on your having decent bandwidth). However, operations such as listing the contents of a large directory, which requires many round trips, are extremely slow. The poor performance here is largely independent of bandwidth and is related to having to grab entire messages instead of being able to use message summaries.
I haven't done any testing where GmailFS opens the same file multiple times and performs subsequent operations on the file. I suspect it will behave badly.
If all of this doesn't dissuade you from giving GmailFS a whirl, have at it and enjoy. Just be sure to visit the GmailFS page (http://richard.jones.name/google-hacks/gmail-filesystem/gmail-filesysteml) to find out what's new and grab the latest instructions and code.
Gmcp (http://mindtrick.net/archives/2004/08/gmail_as_an_online_backup_system_gmcp.php) is a small Perl utility to employ Gmail as a backup service.
There's also a PHP (http://ilia.ws/archives/15_Gmail_as_an_online_backup_systeml)
Gmail Drive Shell
Extension (http://www.viksoe.dk/code/gmail; Windows
only) wraps up the GmailFS into a virtual filesystem visible as just
another drive in Windows Explorer. [Hack #79]
Richard Jones