{"id":102,"date":"2011-02-01T13:04:07","date_gmt":"2011-02-01T13:04:07","guid":{"rendered":"http:\/\/ccgi.ekers.free-online.co.uk\/wordpress\/?p=102"},"modified":"2011-02-01T13:04:07","modified_gmt":"2011-02-01T13:04:07","slug":"backing-up-those-gmail-messages-in-all-mail","status":"publish","type":"post","link":"https:\/\/ekers.co.uk\/index.php\/2011\/02\/01\/backing-up-those-gmail-messages-in-all-mail\/","title":{"rendered":"Backing up those gmail messages in All Mail"},"content":{"rendered":"

I recently read an article on wired.com<\/a> that mentions in passing backing up all of that information you have lurking in the cloud, like emails.<\/p>\n

It got me thinking about a perl script I had for grabbing IMAP data. The basic idea is that the module allows for fetch, delete, copy on IMAP-based email. This allows you, for instance, to grab all of the emails lurking in Gmail’s “All Mail” folder. You know – the place where emails go when you say “archive this one, I don’t want to see it no more.” With the advent of the “Priority Inbox”, using the archive feature, especially with keyboard shortcuts, makes reducing the size of your inbox a much more realizable task.<\/p>\n

Anyway, perl has this module, Net::IMAP::Simple::SSL that lets you access IMAP from perl. Here is a simple script to show how it can do it’s thing. Script comes with no warranty, etc.<\/p>\n

Don’t forget to put your username and password into the script at the top. See $username and $password.<\/p>\n

\n#!\/usr\/bin\/perl\n#\n# example-imap-ssl.pl\n#\n# Get mail from google via imap, and dump it to console.\n#\n\nuse strict;\nuse Net::IMAP::Simple::SSL;\nuse Getopt::Long;\n\nmy $host = \"imap.googlemail.com\";\nmy $username = '<insert your google username>';\nmy $password = '<insert your google password>';\n\nmy $boxes = 0;\nmy $brief = 0;\nmy $dump = 0;\nmy $help = 0;\nmy $verbose = 0;\nmy $mailbox = \"%\";\nmy $remove = 0;\nmy $regexp = \"\";\n\n#----------------------------------------\n# process any command line arguments\nmy $result = GetOptions('help|?+'  => $help,\n\t\t\t'brief|b+' => $brief,\n\t\t\t\"dump+\"         => $dump,\n\t\t\t'dump-folders+' => $boxes,\n\t\t\t\"host|h=s\"      => $host,\n\t\t\t'mailbox|m=s'   => $mailbox,\n\t\t\t\"password|p=s\"  => $password,\n\t\t\t\"regexp=s\" => $regexp,\n\t\t\t\"user|u=s\" => $username,\n\t\t\t\"verbose|v+\" => $verbose\n    );\n\n\nprint \"IMAPS client: $username@$hostn\" if ($verbose || !$dump);\n\nif ($help) {\n\t\tprint \"\n  --brief, -b              synopsis of every message (to,from,subject)\n  --dump                   dump the mailbox selected\n  --dump-folders           dump all folders contained in selected mailbox\n  --host, -h         set the imap server contacted to be \n  --mailbox, -m     \t   set the set of folder(s) to consider\n  --password, -p     the password to use\n  --regexp        display messages matching regex \n  --user, -u         the username to use\n\n\";\n\t\texit 0;\n}\n\n\nmy $imap;\nif ($verbose > 1) {\n    print \"Connect to $hostn\";\n    $imap = Net::IMAP::Simple::SSL->new($host, Debug=>1, use_ssl=>1);\n}\nelse {\n    $imap = Net::IMAP::Simple::SSL->new($host, use_ssl=>1);\n}\n\n$imap->login($username, $password);\n\nif ($boxes) {\n    print \"Enumerating folders for '$mailbox':n\";\n    my @folders = $imap->mailboxes($mailbox);\n    for my $f (@folders) {\n        print \"  '$f'\";\n    }\n    print \"ne='\", $imap->errstr(), \"'n\";\n    $imap->quit();\n    exit(0);\n}\n\n\nif ($dump) {\n    my @folders = $imap->mailboxes($mailbox);\n\n    for my $f (@folders) {\n\tmy $msg_count = $imap->select($f);\n\tfor (my $i=0; $i < $msg_count; $i++) {\n\t    my $msg = $imap->get($i);\n\n\t    if (!defined($msg)) {\n\t\tprint \" undefn\";\n\t    }\n\t    else {\n\t\tfor my $m (@{$msg}) {\n\t\t    print \"$m\";\n\t\t}\n\t    }\n\t} #for (my $i...\n    } #for @folders\n}\nelse {\n    my @folders = $imap->mailboxes($mailbox);\n    for my $f (@folders) {\n\n\tmy $spam = 0;\n\tmy $type = \" \";\n\tmy $msg_count = $imap->select($f);\n\n\tif ($f =~ \/.*spam.*\/i) {\n\t    $type = \"S\";\n\t    $spam = 1;\n\t}\n\tprintf \"%4i messages in $f ($type)n\", $msg_count;\n\n\tfor (my $i=0; $i < $msg_count; $i++) {\n\n\t    my $matched = 0;\n\n\t    if ($verbose > 1 || $brief || length($regexp)) {\n\t\t#print \"$i $type\" if ($verbose > 1);\n\n\t\tmy $msg = $imap->get($i);\n\n\t\tif (!defined($msg)) {\n\t\t    print \" undefn\";\n\t\t}\n\t\telse {\n\t\t    if ($brief) {\n\t\t\tmy ($sub, $from, $to, $date) = \"\";\n\t\t\tfor my $m (@{$msg}) {\n\t\t\t    $sub = $m if ($m =~ \/^Subject\/);\n\t\t\t    $from = $m if ($m =~ \/^From\/);\n\t\t\t    $to = $m if ($m =~ \/^To\/);\n\t\t\t    $date = $m if ($m =~ \/^Date\/);\n\t\t\t}\n\t\t\tprint \"$from $to $date [$sub] n\";\n\t\t    }\n\t\t    elsif (length($regexp)) {\n\t\t\t#print \"regexp: searchn\" if ($verbose);\n\t\t\tfor my $m (@{$msg}) {\n\t\t\t    if ($m =~ \/$regexp\/) {\n\n\t\t\t\t$matched = 1;\n\n\t\t\t\tif ($verbose > 0) {\n\t\t\t\t    for my $n (@{$msg}) {\n\t\t\t\t\tprint $n;\n\t\t\t\t    }\n\t\t\t\t}\n\t\t\t\telse {\n\t\t\t\t    print $m;\n\t\t\t\t}\n\t\t\t    }\n\t\t\t}\n\t\t    }\n\t\t    elsif ($verbose > 1) {\n\t\t\tfor my $m (@{$msg}) {\n\t\t\t    print \"$m\";\n\t\t\t}\n\t\t    }\n\t\t}\n\t    }\n\t}\n    }\n}\n\n$imap->quit();\n\n\n<\/code><\/pre>\n
The new thing I struggled with was the name of the folder where archived emails live. It is<\/p>\n
[Google Mail]\/All Mail<\/code><\/p>\n
But spaces are bad in all kinds of place names, and here is another. There is a wildcard character available ‘%’ and replacing spaces with ‘%’ works fine. So with the name<\/p>\n
[Google%Mail]\/All%Mail<\/code><\/p>\n
all of your mail can be downloaded in one go. When researching this, I also found references to [Gmail] as a the IMAP folder name. YMMV. List the IMAP folders if you get stuck.<\/p>\n
In the same way, sent emails can be downloaded from “[Google Mail]\/Sent Items”.<\/p>\n
Note that the format of the downloaded emails isn’t pretty. There is just one great file with all of them concatenated together. But this is a quick and simple backup. And grep is your friend, too.<\/p>\n
I use the perl above like this:
\n
\n.\/example-imap-ssl.pl --user me --password my_passwd -m '[Google%Mail]\/All%Mail' --dump > 2011-01-31.all-mail.backup
\nbzip2 2011-01-31.all-mail.backup
\n<\/code><\/p>\n
Further information<\/strong>
\nTurning on imap in gmail<\/a>.
\nTurning on ssl in gmail<\/a> (if you haven’t done this, you really should).<\/p>\n","protected":false},"excerpt":{"rendered":"
I recently read an article on wired.com that mentions in passing backing up all of that information you have lurking in the cloud, like emails. It got me thinking about a perl script I had for grabbing IMAP data. The basic idea is that the module allows for fetch, delete, copy on IMAP-based email. This […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[12,15,18,20,21,22,33,40],"class_list":["post-102","post","type-post","status-publish","format-standard","hentry","category-command-line","tag-all-mail","tag-backup","tag-command-line","tag-gmail","tag-google-mail","tag-imap","tag-perl","tag-wired-com"],"_links":{"self":[{"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/posts\/102","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/comments?post=102"}],"version-history":[{"count":0,"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/posts\/102\/revisions"}],"wp:attachment":[{"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/media?parent=102"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/categories?post=102"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ekers.co.uk\/index.php\/wp-json\/wp\/v2\/tags?post=102"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}