Folder name change prompts consolidation of entire library

Troubleshooting
1

About a year ago, I had to move my Tropy project and all my files onto an external hard drive (a 2TB LaCie drive partitioned in half) because I switched to a MacBook Air (from a Pro) and my new computer couldn’t store all the files locally. I rebased the project and the setup worked fine for a year, but recently whenever I do so much as rename a folder on the external drive, Tropy prompts me to consolidate all 10,000+ of my files, which takes hours and overheats my laptop. Today I completed the re-consolidation process, but when I went to print a list, the same thing started all over again (I hadn’t changed any file or folder names this time).

Any suggestions for getting around this problem? It’s really cramping my workflow (is that an expression?) so I’d really like to find a better solution. Thanks!

2

The consolidation takes a long time, because it involves searching for and then analyzing each individual photo. For over 10,000 files that’s definitely not a practical approach especially if you know exactly that only a folder was moved or renamed.

There’s currently no UI for this, but it’s quite easy to update all paths in the project with a single command that takes less than a second, if they’ve changed in a uniform way. You can do this with the sqlite3 tool which should be pre-installed your MacBook. Here are some steps you can take. If you need more help, please let us know the paths where Tropy thinks your files are and the paths where they actually are.

  1. Before you start, please make a backup copy of your project file in case something goes wrong!

  2. Open Terminal.app and cd into the directory of your project file. For example, if your project file is in the Documents folder:

    $ cd Documents

  3. Open the project file with the sqlite3 command. For example, if your project file is called ‘project.tpy’:

    $ sqlite3 project.tpy

  4. Now you should not be at the command prompt anymore, but at the SQLite prompt (the prompt is typically a > instead of the $). To test that you’ve opened the project file, let’s print out a list of all the photos in the project:

    > select path from photos order by path;

    This should print all 10k photos. You can shorten this and print only the first 10 like this:

    > select path from photos order by path limit 10;

  5. If this looks in order, you’re ready to update the paths. For this we’ll use a single update command that replaces all the paths in one go. For example, if your photos used to be nested under a folder /Volumes/OLD/photos/... and that path has changed to /Volumes/NEW/photos/..., then you would update your paths like this:

    > update photos set path = replace(path, '/Volumes/OLD/', '/Volumes/NEW/');

  6. Now print out the list of paths again to check that the paths have been updated correctly.

  7. If everything looks good, close sqlite3 by typing .exit or by pressing Ctrl+D and close the Terminal. Open the updated project in Tropy and test if it finds the photos. To test this, I would select a handful of photos, open them in the viewer, and maybe right-click on a photo in the photo panel and select ‘Show Original Photo’: this should open the respective folder in Finder with the photo selected.

3

Thank you so much for your help! I tried your suggestions but I think I did something wrong.

I updated the paths like this
sqlite > update photos set path = replace(path, ‘/Users/Zso/Downloads/’, ‘/Volumes/LaCie\ Hale/Research\ photos’);

This ran, but when I printed the paths again they pointed to my desktop instead of the external hard drive folder I had specified. I checked in Tropy and it was, indeed, still not finding the photos.

I went back in to update the paths a second time, from ‘/Users/Zsofia/Desktop/’ to ‘/Volumes/LaCie\ Hale/Research\ photos’ and this did not work. I just printed the paths again and it says

Error: no such table: photos

I’m worried I did something very wrong, but I did make a backup of the project in case it’s all messed up.

Thanks again!

4

First off, as long as you have a backup of the file there is certainly nothing to worry about.

The update command above looks fine, but I think you can actually skip the backslashes in front of the spaces there. In any case, instead of trying to figure out what exactly went wrong, I’d suggest to start again by making a new copy of your backup copy and working with that instead. Here are some observations which might help:

The ‘no such’ table error suggests that you didn’t open the actual project file. Note that you can open any file with the sqlite3 path/to/anywhere.tpy command: if the file does not exist, it will be initialized as an empty SQLite database. If you then run a query on the photos table, it would rightly tell you that it does not exist. So, this probably just means that you mistyped the file name or that you were in wrong working directory.

The update command you posted above would not change the path from Downloads to Desktop, so I suspect that some of your files are currently linked to the Downloads folder and some of them to Desktop folder. If that’s the case, you will have to run two separate update commands, like you tried.

Like I said, I would suggest to start over by making another copy of your original project file, open it and print the full list of paths with: select path from photos order by path; In your case, this is a really long list, but because it is sorted alphabetically, it should be easy to scan to determine how many different ‘root’ folders you’re dealing with. Specifically, it’s likely that some of the paths point to your Downloads folder, while others point to Desktop. You should then issue one or two update commands accordingly and print out the whole list again. Again, because it’s sorted it should be easy to scan and ensure that all paths now start with “/Volumes/LaCie Hale/Research photos”.

Finally, when you open the project in Tropy again, make sure that you’re opening the correct copy of the file. At start-up Tropy will open your most recent project at its original location. But this means that if you make a copy of the project file, then edit that copy, then open Tropy, it will open the original file, not the one you changed. To make sure, you have the correct copy open, use File -> Open from the menu and pick out the file you changed.

5

That’s backslashes, not backspaces in @inukshuk’s comments, lest @zsvn be confused. :slight_smile:

Since you’re quoting the path, you don’t need them and in fact I suspect they will result in the change not working.

1 Like
6

Thank you both for your help! Changing the paths separately ended up working, so now all my photos point to the "/Volumes/LaCie Hale/Research photos” folder. However, this didn’t resolve the issue completely, because it was still confused about subfolder name changes within “/Research photos”. I was able to manually consolidate the photos I needed to access immediately, but many more remain this way. Is there an efficient way to locate/print all the missing photos in my project and to resolve them, using the command line? The problem is I don’t necessarily remember what my subfolder names were called before I renamed them (in many cases this was many months ago), and clicking “Show original file” doesn’t seem to work for the missing ones in my library. This makes it difficult for me to provide the path it needs to rename.

7

If these are only a handful of folders that were re-named, I’d try to figure out which on a case-by-case basis and then rename the folders in the database one-by-one. For example, if you find a photo that’s not opening in Tropy, make a note of its filename, then search in Finder for that file to find out it’s current location; compare it to the old one to find out which folder was renamed, and update all photos with a matching path.

For a fully automatic alternative, we have a find photos script. Basically, it works like this:

  1. You point the script at your project file and at a folder, for example at /Volumes/LaCie Hale/Research photos
  2. The script will now scan the entire folder and all its sub-folders and compute checksums for each photos
  3. It will then compare those checksums with all the photos in your project and update the paths for all files where a checksum matches

This script will work whether you renamed folders or even the files themselves.