Consolidating photos afer JSON import

I’m using the JSON import feature to set up a new project with some metadata fields already filled in. This works beautifully, except that my photos don’t render in Tropy unless I consolidate them. This would be fine, except that to make the project manageable further down the line, each photo sits in its own folder. This means I need to click on every single item to consolidate them.

I’ve noticed three oddities:

  • Even though I’ve selected the photos to be linked ‘relative to the project file’ (and also tried ‘relative to home directory’) in the project settings, consolidation only works when using absolute paths (and it also exports JSON with an absolute path, rather than the relative path).
  • If I right-click on a (grey) thumbnail and ask to ‘Show original file’, it has no problem doing that. So the path seems correct (when using absolute paths).
  • Once a photo has consolidated once, I can import it again as a new item and it’ll find the photo, no problem (using an absolute path). If I adjust the path in the JSON to be relative, then it’ll show a colored (not grey) thumbnail.

Why separating each photo in its own folder matters:

  • Each item has one photo associated with it now, but it in the future I’ll be adding more photos per item. (I’m working on a project documenting Roblox games, so each game is one item, and each screenshot a photo grouped with that item.)
  • At the moment, once the first image has been consolidated, all the further photos in the item also show up. This is handy because there’ll be more screenshots than items…
  • I’d use lists instead (one game = one list) but as far as I understand, these can’t be exported without going list-by-list, and I need to be able to export/import easily and not lose the project structure.

Would anyone have any thoughts on what I could do to force Tropy to look into the path provided? Or is there a way to set a parent folder to look into?

When you import via JSON Tropy will create all the items before loading each photo, because the necessary metadata is already there in the JSON file. The JSON import already schedules all imported items for consolidation, so while it’s true that the photos won’t show up immediately as they are imported, they should get consolidated shortly afterwards without you having to do anything else. Now, this will only work if Tropy can find the images of course. Whether or not that’s the case is unrelated to the project and how it is configured to link the photos, it only depends on the JSON file.

Basically, if you have absolute paths or URLs in the JSON file, then Tropy should be able to resolve them as they are. If you have relative paths in the JSON file then Tropy will resolve them relative to the JSON file. The easiest example is a JSON file that references all photos by file name only: importing those files will work as long as the photos and the JSON file are all in the same folder.

In your case, do you keep all the folders representing an item/game in a common root folder? If so, I think the best setup is to put the JSON file in that folder and then use relative paths for each photo.

And for future portability of the project, I’d also set the project to use relative paths and put the project file in the common root folder. This way, the JSON file and the project file would be in the same folder and you’d actually have the same relative paths in the JSON files as the paths that Tropy will be storing in the project file. The benefit of setting it up this way is that you’ll be able to move the entire project folder without having to worry about breaking the file associations.

Hi Sylvester,

Sorry for the delayed reply!

You’ve confirmed my understanding of how it works, which is why I thought my issues were so odd. But now I’ve taken a look at the log (should really have started there), I get this error when it tries to render an image: “Error: EPERM: operation not permitted”

All my files were on a OneDrive folder; I moved the whole project (project file + images) out and ta-da, problem solved.

Would you have any tips on getting it to work with cloud storage?

I think this could either be an outright permissions issue or something related to the way OneDrive syncs the files. Do you have any special permissions set on the folder that could make Tropy ineligible to accessing the files in it? Otherwise, I’d check if the files are sync’ed fully. OneDrive has a feature where files are loaded on demand, but I’ve seen plenty of issues with this in the past: sometimes Tropy will try to access the file from the file system, but will receive an error before OneDrive fetches it. To see if that’s the case you could try importing a single file from the OneDrive folder for test reasons, making sure it’s definitely there on the local drive (e.g., try opening it from Windows Explorer first or make sure OneDrive is configured to keep a local copy at all times).

You’re right – it’s a permissions issue. I’ve set the folder to ‘Always keep on this Device’ so there’s definitely a local copy, but that doesn’t fix it. It’s not a problem, however, if I’ve consolidated the item in the past (even if I delete all items and import the JSON again). Any thoughts on what I could do with the permissions issue?

A quick related question: Is it correct that the JSON export contains the absolute photo paths, rather than relative paths (even if the project settings are set to relative)?

Yes, the JSON data contains always the absolute path to the file. The reason for this is because the JSON data is also the basis for copy/pasting items (e.g. from one project window to the other) and is therefore not necessarily rooted in a specific file location. I think we’d be open to adding options to control this, but I’m not sure it would be worth doing so since the paths can be adapted relatively easily by a plugin or whoever consumes the JSON (for example the way the archive plugin turns them into relative paths).

Regarding the file permissions, this sounds a little bit like file access is frequently handled by sandboxed apps nowadays: that is, the OS gives the application access to the file if you selected it through a file picker or via drag and drop but not otherwise. You mentioned that it works if you move the files out of the OneDrive folder, right? In that case, do you notice any differences if you compare the file properties of the file copies in and outside of the folder?

Makes sense with the JSON paths – I don’t think it’s a problem at all, it’s easy to adjust. Just wanted to make sure nothing strange was happening on my end.

I’ve checked the file properties, and their respective permissions specifically, and they are the exact same… Messed around with permissions a little anyway but that made no difference. So strange!

I think I can move forward knowing that each item just needs to be consolidated once, it’s fine for this project. But happy to help however I can if this is something you’d like to look into at some point.

Which operating system are you using? I’d have a few more ideas that we could try if it’s Linux or macOS but I was assuming this is Windows 11?

So sorry for disappearing! I’m just seeing your message now; will pay better attention to my notifications.

I’m on MacOS Sonoma 14.3.1.

Ah, OK, on macOS there are a few more things we can try. First off, it’s most likely that there some extended file attributes in play that make the files behave differently in vs outside of the shared folder. On the Terminal you could check the extended attributes using the xattr -r command with the path to the folder where the files are. I’d guess that there is com.apple.quarantine or a similar attribute involved at some level. Do you see any difference between the two locations looking at the extended attributes?

Instead of looking at the file attributes it could also be a permissions issue. You could try giving Tropy full disk access in the security preferences.

Giving Tropy full disk access fixed it :face_with_peeking_eye: Thank you!