Hi,
I recently had to change my computer and I moved all my Tropy projects to the new one.
I replicated the exact same folder structure. However, Tropy needs to re-consolidate everything and it crashes every few items. I have to open and re-open every single process hundreds of times scrolling the items list in order to reconsolidate fewer items per time (it still crashes every 2 or 3 groups).
Where’s the problem here? Can I do something to fix it?
Thanks!
Here the two last lines of the crash logs
{"level":40,"time":1689151872282,"type":"renderer","name":"project","query":"UPDATE photos SET checksum = $new_checksum, color = $new_color, filename = $new_filename, mimetype = $new_mimetype, orientation = $new_orientation, page = $new_page, path = $new_path, protocol = $new_protocol, size = $new_size WHERE id = $id","ms":282,"msg":"SLOW: db query took 282ms"}
{"level":40,"time":1689151872562,"type":"browser","name":"main","msg":"project[1] render-process crashed: crashed"}
Oh no, I’m sure this is the issue we’ve been seeing on some Linux systems with the bundled libvips build. It happens when importing multiple images in a set which consolidation essentially is – but worse because you can’t really make the sets smaller in your case.
Do you still have access to your original computer / hard drive? If so, you could just copy all the thumbnail data over to the new one and so consolidation won’t be necessary anymore. On Linux the cache is typically at ~/.cache/tropy/. There is one folder per project; you can either copy all of them or only for that one project. To find out the right folder name you can check your project id (e.g. in the project preferences) or just look at some of the images in the folders.
Otherwise I’m afraid there’s no quick solution; we need to replace the bundled libvips with a new build which doesn’t have the issue, but testing this is time consuming because it doesn’t crash everywhere (it must be related to the versions of other linked libraries installed on the system).
Can you check which version of libvips is provided by your distro? If it’s version 8.14.x we could look into creating a build for you that uses the system libvips instead.
Yes, it won’t automatically pick up the system libvips we’d have to use a custom version of Tropy for this. Unfortunately 8.12 is not recent enough. If you could install version 8.14.2 we could try with a Tropy 1.15 beta using the system-wide library.
Otherwise you’d have to downgrade to Tropy 1.12 instead (this is bundled with a different version of libvips where we have had not reports of the issue).
We’d love to make a patch release if we can figure out the reason for the crash and a way to fix it.
Could you try to run this script to see if it reproduces the crash for you? If you download and unzip this Gist you should be able to test it like this:
node -v
This is just to make sure you have Node.js installed (if not you can install it with apt). Then, to install sharp and Tropy’s libvips build run:
SHARP_IGNORE_GLOBAL_LIBVIPS=true npm install
This should install everything. To make sure that Tropy’s version was installed you can search e.g. for ‘poppler’ the PDF library which is in the Tropy build but not in the default one:
grep -r poppler node_modules/sharp/vendor
If this finds a match (particularly in THIRD-PARTY-NOTICES.json) then the right build was downloaded.
You can run the test script on any directory containing JPG/PNG files. It would be interesting to know if this script crashes for you and if so, how many files it takes to make it crash; if it does we may be able to narrow the issue down further. For reference, on my Linux system this runs fine processing hundreds of images in parallel.
You can run the script like this:
node . ~/Pictures
Instead of ~/Pictures you can put any path to a directory with JPG/PNG images (i.e., put node . followed by a path to your images). For example, you could point it to the folder containing your project’s images.
If this produces a crash or error, please let me know how many images were processed. If that’s a large number it would also be great if you could narrow it down since this script isn’t very smart so it may legitimately run out of memory for lots of images (Tropy will not keep more than a handful of such images in memory at the same time).
I installed node.js and npm and what I get from node . ~/pics is:
$ node . ~/pics
file:/// [...] script/index.js:12
for (let file of await readdir(wd)) {
^^^^^
SyntaxError: Unexpected reserved word
at Loader.moduleStrategy (internal/modules/esm/translators.js:133:18)
at async link (internal/modules/esm/module_job.js:42:21)