Free program for finding duplicate photos in the Apple Photos App on OS X

UPDATE: This script is no longer necessary given Apple’s introduction of automatic duplicate detection to iOS. I’ll leave this post here for archival purposes, however.

If you have a huge Photos library on the Mac, you very likely have a bunch of duplicate photos. People have also found that duplicates photos are sometimes created when you upgrade from iPhoto or Aperture to Photos. Unfortunately, if you look at the “free” programs on the App Store, you find that they are generally either scams or teasers for a paid version.

So, I wrote an AppleScript to go through every photo selected in Photos and create a new Album with just the duplicate photos found. (With a large album I recommend selecting one or two years at a time to avoid issues with memories.) The script can be copied to your Photos script folder (if you have the script menu enabled) or you can just run it from with ScriptEditor. I also created a small script to reveal the ID of a selected photo, which can be useful for debugging the library and/or finding an original file within the inscrutable Photos database. Most people (if anybody) will just want the Find Duplicates script. Download them at the link below:

Photos AppleScripts

Once the script finds any duplicates, it creates a folder called “Duplicates” and adds a new album with the found duplicates. You should then go through that album and confirm that the duplicates are legitimate. If they are, select which the one duplicate you want to remove (you won’t want to get rid of the entire pair, presumably), right click and select delete. Make sure you don’t delete both in the pair, but if you do it’s not the end of the world; they will stay in the deleted photos album for a month.

Let me know if something doesn’t work right. The Photos AppleScript library doesn’t allow any destructive operations other than deleting albums (which I don’t do in either script) so the worst case should be that if it doesn’t work it just doesn’t find duplicates that it should.

As a couple folks pointed out in the comments, the next step will be to create a script that allows you to “merge” a pair of photos, moving all the album associations and tags onto just one. I haven’t looked into whether this can even be accomplished with the AppleScript hooks available in Photos, so if anybody has an insight I’d appreciate it.

17 thoughts on “Free program for finding duplicate photos in the Apple Photos App on OS X

  1. Pato

    Hi,
    I found your script repository online, and since you have already worked with Photos.app and presumably know your stuff, I wanted to make a little request, since I cannot script myself.
    My annoyance is that it is no longer possible to right-click pictures and reveal them in Finder in Photos.app. This guy wrote a couple of scripts (http://www.mac.si/posts/photosreveal.html) that should be able to accomplish that, but I can’t get them to work and can’t track him down.
    I would be glad to donate to a worthy case of your choosing if you could see your well to help me out.
    Thanks,
    Pato

    Reply
  2. pam

    Hi,
    I ran from script editor and zero duplicates were found. I think I enabled script menu.
    How do I get it into the right folder to run in my Photos?

    Thank you.

    pam

    Reply
    1. Jonathan Post author

      If you enable the scripts menu bar item (as discussed in the post) go to Photos and then click on the scripts menu (it’s actually an icon, not the actual word). There should be a menu item that will open the right folder for you.

      Reply
  3. Anders

    This duplicate script of yours simply compares the filenames and consider them as duplicates if they have the same filename. Seriously? What if I have a camera that resets the counter or two cameras sharing the same filename serie? This is simply a really bad idea.

    Reply
    1. Jonathan Post author

      First, the script only checks for similar file names occurring within a few photos of each other for exactly that reason (as well as speed). Second, it doesn’t delete a damn thing. It simply creates a photo album with the possible duplicate photos so you can go through them quickly and decide if they really are duplicates and delete them yourself if you want.

      By the way, reading for content takes a bit of time, but not nearly as much as writing comments based on mistaken understanding.

      Reply
  4. Sherwood Botsford

    Filenames are a bad criteria, even if limited to a a span of a few pix either way.

    1. As time goes on, you end up with multiple versions that can have the same name. E.g. You have a black and white version, a sepia toned version, and a sharpened version. You have a version for a web page,

    2. As time goes on you have different photos that move to different projects. (I’m an Aperture user.) I keep a major subdivision for business, family and holidays. I may also keep a folder of gems, a folder of ones I want to sync with my phone.

    3. What happens if I edit a pic with photoshop flatten it back to jpeg and import it?

    4. If I have duplicates, what happens to the tags in the one I chuck.

    ***

    A better system:

    * On import the plain file is given a unique id — MD5 sum of the unmodified file. This sum is encoded into one of the EXIF or IPTC fields, preferably one that isn’t wrecked by photo editing software.

    * On creation of a duplicate of this ID has a suffix added. This is true in general. A new version gets a copy of the old version’s id + a letter or two. If a file is handed off to an external progrm it’s given a suffix for the new copy, and a note as to the program.

    * In some cases you end up with 2 new files: E.g. if you send your file to photoshop, you may get back both a .PSD file and a .JPG file. These should get different internal suffixes.

    * Done right, this can provide a trail of deriviatives stretching back.

    Reply
    1. Jonathan Post author

      I agree filenames aren’t a great way to find duplicates, Sherwood, but I think what I’ve done is about all one can do with a simple script. The possibility for errors is why my script doesn’t delete anything but just puts references to the potential duplicates in an album for user review. That seems to solve all the duplicate cases I’ve found. Ideally one would search by image content, but that would be quite an effort and could not be done in a simple script.

      That said, almost none of your cases apply to Photos. Multiple versions don’t actually happen in Photos the way you suggest. Those are kept hidden and attached to a single image from the perspective of the user and the database. In fact, several of your points apply to Aperture, not Photos. This is a script that ONLY works with the Photos app.

      If you edit a pic in Photoshop and then reimport, it will likely end up non-adjacent to the original picture in the database, unless you happen to edit the last picture you took, so the span filter wouldn’t find it. But it’s a potential issue if you don’t bother to change the name of the photo.

      Your solution would probably work, but it would be on Apple to adopt it. I doubt the right hooks are available in Photos or Aperture for a plug-in or third-party program to accomplish it.

      Reply
  5. Robert

    Hi,

    given the following situation:

    There is a duplicate pair of a photo, one is part of album A and one of album B.

    If I now delete the duplicate in album B, the foto doesn’t show up in album B anymore, right? So I have to “re-reference”/insert it manually again, don’t I?
    So if I had 300 albums with duplicates of one foto, I would have to re-insert the original to all 300 albums manually?

    I didn’t find this feature in any solution on the market…
    Help anyone?

    Reply
    1. Jonathan Post author

      Robert:

      Thanks for writing. Yes, that is definitely a problem my script does not help solve. I’ll have to think about how one would solve that and if it would even be possible to do from Apple Script. Maybe a merge function would be possible where you select two photos and have any reference to the second in an album replaced with a reference to the first. If I get some time I’ll look into whether that could be done. This could be part of a second stage script that actually did the deletion of the second photo.

      Reply
    2. JW Andersen

      Albums are only a convenience item for the user, all photos reside in the “Photos” Library. Only one photo actually exists.

      You can have that single photo in every ‘Album’ within your photos application. They are only pointers to the real picture. This is the same in Music.app, TV.app & Books.app..

      One file, many locations.

      Reply
  6. Rob

    Thanks for posting this script. It came in very handy to help out a friend who had thousands of duplicate versions of photos with the same filenames within 20 or so spaces of each other (I opened up the search criteria a bit). I eventually noticed that one version had the metadata of the original and one was a smaller file with just the photo itself so modified your script to identify the smaller file and put them in an album for final review and deletion. Saved her many, many hours. Thanks!

    Reply
  7. David

    Thanks a bunch. I had been working on the same idea in AppleScript, also comparing to just nearby duplicates, and also just adding duplicates to a “redundant” folder. I’m using the size and date as matching criteria.
    However, my script was impractical, because “get” ing a photo via apple events took about 2 seconds for each photo. Your idea of getting the selection in one fell swoop was critical.

    Reply
  8. Jonas

    Hello Jonathan
    Thank you for the Script!
    It helped me a lot. While other dedicated apps couldn’t figure out the duplicates, your script found them all 🙂
    Jonas

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *