How to Undelete Files in Amazon S3

While S3 is a great storage platform, what happens if you accidentally delete some important files? Well, S3 has a mechanism to recover deleted files, and I’d like to go into that in this post.

First, make sure you have versioning enabled on your bucket. This can be done via the API, or via the UI in the “properties” tab for your bucket. Versioning saves every change to a file (including deletions) as a separate version of said object, with the most recent version taking precedence. In fact, a deletion is also a version! It is a zero-byte version which has a “DELETE” flag set. And the essence of recovering undeleted files simply involves removing the latest version with the “DELETE” flag.

This is what that would look like in the UI:

To undelete these files, we’ll use a script I created called s3-undelete.sh, which can be found over on GitHub:

(If you like to hang out on StackOverflow, this is the answer that inspired my script: http://superuser.com/a/1019940)

Next, before we can run that script, the AWS CLI will need to be installed. You can get it from https://aws.amazon.com/cli/. Then you must run aws configure to set up your credentials.

The final step is to be able to run this script is to make sure that your credentials have the appropriate policies in IAM. Following the example of the previous post, we’ll need to have s3:ListBucketVersions and s3:DeleteObjectVersion added. So the final policy will look something like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketVersions"
            ],
            "Resource": [
                "arn:aws:s3:::yourusername-odrive"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:DeleteObjectVersion"
            ],
            "Resource": [
                "arn:aws:s3:::yourusername-odrive/*"
            ]
        }
    ]
}

Long post is long, but hang in there, we’re almost done!

Let’s try running this script against the bucket named “yourbucket” looking for deleted files matching the string “test.*txt”:

$ ./s3-undelete.sh yourbucket test.*txt
aws s3api delete-object --bucket yourbucket --key "temp-holding/test.txt" --version-id y8yX9qmgrtdKR3TawewQnm9tcH1RVPZk;
aws s3api delete-object --bucket yourbucket --key "temp-holding/test2.txt" --version-id xElabQ.YxzIVK6TWlQ76kzYmqRxRiLwj;

Hooray! Two deleted files were found. The output is AWS commands that you’ll need to run to undelete those files (by removing the revisions that set the delete marker). So let’s run the script again, but save these files to a list:

$ ./s3-undelete.sh yourbucket test.*txt > files.txt

And now send all of those undelete commands off to Bash:

$ cat files.txt | bash
{
    "VersionId": "y8yX9qmgrtdKR3TawewQnm9tcH1RVPZk", 
    "DeleteMarker": true
}
{
    "VersionId": "xElabQ.YxzIVK6TWlQ76kzYmqRxRiLwj", 
    "DeleteMarker": true
}

And finally, take one more look at that bucket:

$ aws s3 ls s3://yourbucket/
2016-12-11 16:50:51       1387 test.txt
2016-12-11 16:50:55       1387 test2.txt

The files are back, and there was much rejoicing.

This just scratches the surface of what you can do with Amazon’s S3 storage service. There are so many more neat things you can do with S3, such as hosting a static website or even distributing your files via BitTorrent. I encourage you to check them out!