I was given about 7,000 images to categorize and reorganize as a part of a research project, as well as their metadata in a text file. I can change the format of the file, of course, but I’m still unsure of what to change it to.
I have no prior experience using Tropy, so I am unsure of how to proceed. Any and all help is appreciated!!
You want either a script or plugin to turn the text file into a JSON file which Tropy can import (CSV is also an option if you prefer to work with spreadsheets, but it’s less flexible and therefore might not work well with your data). How easy or difficult this is to do, will depend mostly on the way the file is currently structured.
The first question I’d ask is how these 7,000 images should be mapped to items in Tropy. That is, do you want to turn these into 7,000 items, each with a single photo and the corresponding metadata, or are these going to be a smaller number of items with multiple photos?
As for the metadata in the file, if you post an example of the how the metadata is structured (and how it is linked to the photo file) we’ll be able to give you better advice.
Thank you so much for your response!
There will be a slightly smaller amount of items, some items will have multiple photos, while others will just have a single photo and corresponding metadata.
Here is an example of how the metadata is currently structured in the text file:
Artist Artist Dates Date Description Dimension: Height (cm) Dimension: Width (cm) Filename Keywords Location Medium Medium - Technique Path Title
Netherlandish Artists ca. 1525 from west end of church Aarschot Jube - AJ01.JPG microarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:(name):Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ01.JPG Aarschot Jube
I also converted the text file into a JSON file, if that would be helpful? Please let me know if you need anything else from me!
Thanks! From the example above I’m still not quite sure of the syntax of your text file. In order to parse/re-format this it’s crucial to know how and where the separators (line breaks, spaces, commas etc.) are.
You can enter preformatted text here by wrapping it with triple backticks (```) or by using the respective button in the editor toolbar. Otherwise it’s hard to tell where the line breaks are.
Thank you! Please let me know if this is any better to read!
Netherlandish Artists ca. 1525 from west end of church Aarschot Jube - AJ01.JPG microarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ01.JPG Aarschot Jube
Netherlandish Artists ca. 1525 full view Aarschot Jube - AJ02.JPG Aarschot JubeAarschot Jube - AJ02.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ02.JPG Aarschot Jube
Netherlandish Artists ca. 1525 left third Aarschot Jube - AJ03.JPG Aarschot JubeAarschot Jube - AJ03.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ03.JPG Aarschot Jube
Netherlandish Artists ca. 1525 centre third Aarschot Jube - AJ04.JPG Aarschot JubeAarschot Jube - AJ04.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ04.JPG Aarschot Jube
Netherlandish Artists ca. 1525 centre third, detail Aarschot Jube - AJ05.JPG Aarschot JubeAarschot Jube - AJ05.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ05.JPG Aarschot Jube
Netherlandish Artists ca. 1525 right third Aarschot Jube - AJ06.JPG Aarschot JubeAarschot Jube - AJ06.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ06.JPG Aarschot Jube
Netherlandish Artists ca. 1525 right third, detail Aarschot Jube - AJ07.JPG Aarschot JubeAarschot Jube - AJ07.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ07.JPG Aarschot Jube
Netherlandish Artists ca. 1525 oblique view Aarschot Jube - AJ08.JPG Aarschot JubeAarschot Jube - AJ08.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ08.JPG Aarschot Jube
Netherlandish Artists ca. 1525 west vaulting, north to south Aarschot Jube - AJ09.JPG Aarschot JubeAarschot Jube - AJ09.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ09.JPG Aarschot Jube
Netherlandish Artists ca. 1525 west vaulting, south to north Aarschot Jube - AJ10.JPG Aarschot JubeAarschot Jube - AJ10.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ10.JPG Aarschot Jube
Netherlandish Artists ca. 1525 west vaulting, north corner Aarschot Jube - AJ11.JPG Aarschot JubeAarschot Jube - AJ11.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ11.JPG Aarschot Jube
Netherlandish Artists ca. 1525 west vaulting, centre third Aarschot Jube - AJ12.JPG Aarschot JubeAarschot Jube - AJ12.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ12.JPG Aarschot Jube
Netherlandish Artists ca. 1525 east vaulting, centre third Aarschot Jube - AJ13.JPG Aarschot JubeAarschot Jube - AJ13.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ13.JPG Aarschot Jube
Netherlandish Artists ca. 1525 west vaulting, south third Aarschot Jube - AJ14.JPG Aarschot JubeAarschot Jube - AJ14.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ14.JPG Aarschot Jube
Netherlandish Artists ca. 1525 east vaulting, south third Aarschot Jube - AJ15.JPG Aarschot JubeAarschot Jube - AJ15.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ15.JPG Aarschot Jube
Netherlandish Artists ca. 1525 choir side, oblique view Aarschot Jube - AJ16.JPG Aarschot JubeAarschot Jube - AJ16.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ16.JPG Aarschot Jube
Netherlandish Artists ca. 1525 choir side, centre third Aarschot Jube - AJ17.JPG Aarschot JubeAarschot Jube - AJ17.JPGmicroarchitecture Belgium, Aarschot, O-L-V Micro-Architecture Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aarschot Jube:Aarschot Jube - AJ17.JPG Aarschot Jube
Pieter Aertsen Amsterdam, 1508 - Amsterdam, 1575 1559 Aertsen, Pieter - BMM01.JPG Belgium, Brussels, Musées Royaux des Beaux Arts de Belgique Painting Macintosh HD:Users:tiannauchacz:Pictures:Work:Images - Monuments:Aertsen, Pieter:Aertsen, Pieter - BMM01.JPG Christ in the House of Mary & Martha```Thanks!
So, by the looks of it you each line references exactly one image file, yes? I think in this case it might be easiest to convert your file into a suitable CSV and import it via Tropy’s CSV plugin. You can probably convert this using a Libre Office spreadsheet (or Excel) or good text editor (with find/replace functionality). To get started, I’d recommend creating Tropy items by hand for a couple of lines (i.e., import the referenced photos and copy the metadata to the corresponding fields in Tropy) and then export those items with the CSV plugin. This will basically give you a template of how you need to convert the text file.
Thank you! I’ve gotten to the point where I’ve created my own metadata template, with each of the corresponding fields that were in the original text file. Everything looks fine in Excel, but when I go to import the CSV into Tropy, I get the error: “Invalid Opening Quote: a quote is found inside a field at line 2”
I can’t find anything wrong in the line; have you seen this error before? Or know what it is referencing so I can look for it?
Thanks again!
Either there’s some syntax or text encoding issue. Could you post the file so we can take a look?
This already looks promising, but it isn’t formatted in a way that the CSV plugin can understand yet. Please create a sample item in Tropy first and export it using the CSV plugin. This will give you a working template (including the header line) of what you need to convert your data to.
You will also need to re-format the file paths before they can be imported. Currently, you have : instead of / and the leading “Macintosh HD” will also need to be removed (this should be easy to do with search and replace patterns in Excel).
There are also a bunch ASCII group delimiters in the data. Is your original input file separated using ASCII delimiters? That might be helpful to know for the conversion. In any case, in the final CSV these delimiters are potentially problematic and I’d suggest to remove them in the conversion process.
Thank you! While working on the edits you’ve shared, I’ve come to the issue where the delimiter field in the CSV plugin won’t recognize tab as a delimiter. I have tried “< tab >” (without the spaces) as well as “\t”, but the headings in the exported sample CSV contain those characters rather than the actual tab character. Does this mean that it does not recognize tab, or does the heading retain the non-character form?
I’m also not seeing any ASCII group delimiters that I could remove either, could you please provide me an example of what one looks like in my data?
Please let me know if there are any additional files you need! Thanks again!
To use tabs as the delimiter you’d set the option to \t however the UI currently escapes the backslash. If you absolutely want to use tabs the quickest way to do it is to set the delimiter to \t in the config, then close the preferences window. Select “Help > Show plugins folder” from the menu and open the config.json file in a text editor. In the file, find the plugin configuration where delimiter should be set to to \\t and remove the extra backslash. Save and close the file, reload Tropy, and if you try exporting your sample item you should see tabs used as the delimiter.
Differentiating between tabs and other white space characters can be difficult though. I’d really suggest to use a comma, semicolon or another suitable delimiter instead.
How the ASCII group delimiters are displayed will similarly depend on your text editor. If you haven’t noticed them, I’d guess that they are shown as white space too.
In my text editor they are displayed as unprintable characters (the ^]). I assume they were used in the original text file.
Sorry for the radio silence! I’ve gotten better metadata and now have a better understanding of the plugin (I thought it attached metadata to existing photos but now I know what it does).
I’ve gotten the export to look the way I expect (I had it export tags, quotes, etc. on accident), and it successfully can import CSVs that I’ve exported. However, any modification of the CSV to try to create new rows seems to confuse the plugin and make the headers not work.
Here is the error:
“Ensure all headers are valid metadata property IDs. Unknown properties:” (it then lists all of the properties I have in my template)
Any and all help would be appreciated!
This sounds odd: adding rows should not impact the interpretation of the header at all. Just to be sure, you’re exporting a CSV file; importing it back works, but once you add more rows it fails? Could you post an example of the CSV data that fails to import?
The plugin throws the particular error message when it finds properties that are unknown to Tropy. Typically, this would happen if you’ve installed additional vocabularies and export an item using properties from one of these vocabularies. If you later import that item to a different Tropy installation where the vocabularies have not been installed yet, the plugin would show this error message. Tropy can handle unknown properties, but in most cases you’d definitely want to install the missing vocabulary, that’s why the plugin raises an alert in this case.
Thank you! Yes, I’m exporting a CSV file to get the template header, and that works. Whenever I change the data, even a miniscule part, it throws the error. The only change in this file from the original export is the word ChangedData in the second row, where Title was originally. I’ve attached the CSV file for that.
I haven’t installed or uninstalled any additional vocabularies, I’ve just been using the ones available in the template creator.
ChangedData.csv (708 Bytes)
Thank you again for all of your help!
How are you changing the data? In the file you posted both rows are wrapped in quotation marks. This means that the file has only a single column in each row. Since Tropy definitely did not export the data this way I’m assuming they’re added by your text editor.
I edited it in Excel! The quotes don’t show up in Excel but do in Notepad.
