"Watch folder" not always imported

I’ve got a watch folder set and images import only part of the time. I’ve been trying to work on two computers, with the project file and its folders on dropbox. Sometimes I put new files in the assets folder and they appear in the project right away. Other times they don’t show up. Out of 350 files, 158 have imported. The files are all TIFFs, JPGs or PNGs.

An update - on the last batch I tried importing one of the missing files individually and got this error message:

{“msg”:“Failed to import item.”,“stack”:“Error: source: bad seek to 33189672\nsource: bad seek to 33189672\nsource: bad seek to 33189672\nsource: bad seek to 33189672\nsource: bad seek to 33189672\nsource: bad seek to 33189672\nsource: bad seek to 33189672\nsource: bad seek to 33189672\nsource: bad seek to 33189672\nTIFFFillStrip: Read error at scanline 348; got 2428 bytes, expected 144338\ntiff2vips: read error”,“system”:“Windows_NT 10.0.22631 (x64)”,“time”:1715641919599,“version”:“1.16.2”}

I suspect that the files become visible on the file system before they are fully synced. We can look into adding longer pause between the time the file is detected and imported.

Slightly unrelated, but is your watch folder the project’s asset folder? For Tropy’s standard project type, Tropy assumes that it has full control over the asset folder, so it is not expected to add files manually to that folder. On import, Tropy would store a copy of each file in that folder, so the intended setup is for the watch folder to be outside the asset folder and files you add there would get copied to the asset folder. Obviously this requires extra space and you can in fact use a watch folder that’s identical to your asset folder, but it would be better to use an advanced project for this kind of setup. Advanced projects will not create a new copy of each file, but instead use the file at its original location.

1 Like

thanks inukshuk, this is helpful. What is an “advanced project?”

When you create a new project with Tropy you can choose between the advanced and standard project types. The two are mostly the same thing, but the standard project type creates a full project/assets folder and is configured with certain assumptions in mind:

  • The standard project there is always a .tropy folder with file project.tpy and an assets folder
  • It will copy and rename every photo you import into the assets folder and use relative paths to reference the files
  • This means that in practice it should never be necessary to consolidate a project, because everything should stay in one place together (on macOS the project folder is also bundled, meaning it can be treated just like a single file by most apps)
  • The project database should perform a little better because it uses an improved journaling mode that uses an additional file (which is OK in this case because the whole project is contained in an extra folder)

The advanced project gives you a little more flexibility, because it creates only the project database file. It doesn’t copy or rename the files during import, so you can decide how and where to store the photos. For example, if you want the files stored in a specific structure (folders, names, etc.) or in several different locations you’d need to use an advanced project. The downside of the advanced project is that you need to be aware of the relationship between the files and the project file and it has implications when you want to move the project around or open it from different devices and so on.

thank you inukshuk, I suspected that there were advanced & standard project types, but due to a bug I’m experiencing I’m cut off from that option.

Maybe this should be a separate bug report, but this is what’s happening when I go to Tropy Preferences / Project:

Some of the entry boxes and drop-down boxes are cut off. I can’t change the Project type or do anything that utilizes the part of the menu that’s cut off.

The window doesn’t let me drag on the edge to change its size, or maximize it by clicking on that upper-right box icon.

Windows 11 Home, Version 23H2, OS build 22631.3593

I set up a folder that pointed to in Preferences as the “Watch folder” and copied all of the files from the assets folder to that folder. What happened was that a number of additional images imported. But there’s still a lot that didn’t. I’m up to 207 items out of 355. I tried manually importing a bunch of the files that didn’t import before. They didn’t import. I added a new file to the Watch folder. That one imported automatically, right away.

When I try to import these unimported files, I don’t get any error message. I know some or all of them are tifs. I don’t know how to generate a list of the ones that didn’t import. When I order the items by title, Tropy orders them differently from Windows Explorer - a lot of them are long strings of numbers and letters. So it’s hard to systematically figure out what could be stopping these files from importing.

Yes, the preferences window certainly doesn’t look like it’s supposed to. Would you mind sharing your log file with us? You can reveal it via the menu Help > Show log files. This might help us because it will include details about the resolution, scale factor and so on.

The watch folder detects files as they are added to the folder or, if Tropy is turned off, since the last time it was up and running. Each file is then scheduled for regular import. Under settings, there is a related option that controls how duplicate files should be handled during import. I suspect that in your case it is set to ignore duplicates and that those files which doe not import automatically are already present in your project. You can also try to import those files again and check your log file: if any of the files are ignored, there should either be error messages there or a notice that they are being skipped because the images already exist in the project. To determine if a file is a duplicate, Tropy looks at the file contents, not the names.

In general, I would use the watch folder differently in a standard project type versus advanced projects. In a standard project, it’s a good practice to watch a folder where your incoming images are placed. For example your Downloads folder. When images are added to that folder they will be automatically imported into Tropy (and copied to the project folder). The images also remain in the Downloads folder in case you need them for anything else, but you can also delete them there eventually and they will still be in your Tropy project.

In an advanced project, it makes more sense to watch the folder where you actually want to keep your images. When you put a file there, it will be added to the Tropy project but Tropy expects the file to stay there going forward. If you move or rename it, the file will have to be consolidated in the project.

hi inukshuk,

Yes, I formerly had the assets folder as the watch folder and then I changed that, and copied all the files into it. The assets folder had a bunch of items that didn’t import because I’d been putting files for import into it. Does it only recognize files that have been added since the last time the project was open? Maybe I need to empty the watch folder and start adding files again?

I did change the settings to ignore duplicates. Since I’m having to visually scan 150-odd names it’s possible I’m missing some duplicates but that doesn’t account for specific files I’ve looked for. There’s a group of files that I don’t find in the items. And I do know there’s a certain number fewer items than files as itemized earlier.

tropy.log (32.7 KB)

I just tried importing 6 files. A window said “importing 6 files”… but I don’t see any of them in the Items, the number of items didn’t change, and there’s no new files in the “Last import” list. Maybe you can see what happened in this updated
tropy.log (36.6 KB)
log.

According to the log file all six photos were indeed skipped as duplicates. This explains why the number of items didn’t change and why there are no new items under last import. It means that you already have those photos somewhere in your project (they could have a different name but the images themselves are identical). If you don’t see them in the project view, it’s also possible that they are still in the project under deleted items.

The watch folder semantics is as follows: while you have the project open the folder is watched continuously and each new image file added is scheduled to be imported. When you close the project, the watcher is stopped as well. Every time you open the project, the folder will be scanned for files which were added since the last time the watcher was running.