Date sorting / date recognition


#1

It seems that date fields are actually considered as alphanumerical fields and not as proper date fields.
Is it scheduled to install “something” that would enhance date recognition (something like as what exists in Zotero where the software recognizes the difference between yyyy-mm-dd or dd-mm-yyyy when the field is filled) or should it be better to insert all dates in a yyyy-mm-dd format?


#2

Currently, all metadata fields are basically strings, but we are planning to support more specific types (most notably dates and names) with special features and input widgets. For dates, we will be using the upcoming extensions to ISO 8601 (previously EDTF) to support a wide range of date features such as uncertain/approximate dates, date ranges, intervals etc. (if you are interested about the specifics, take a look at edtf.js which was written specifically for use in Tropy). All date values will go into a special index which will allow us to support efficient time range searches in Tropy (probably using an interactive timeline, but we have not finalized a UI yet) and we want to provide a dedicated widget to make input easier. We hope that we’ll be able to support sorting using date columns, but that will be very tricky (because of custom templates, it is not guaranteed that fields like dc:date are indeed date values); this does not apply to special fields such as the creation and modification dates, which will definitely be sortable as dates.

While date fields are not fully implemented yet, you can already benefit from using valid EDTF values in date fields (e.g., ‘2017’ or ‘2017-12~’), because they will be formatted based on the current locale. For example, ‘2017-12’ would be formatted as ‘déc. 2017’ in the upcoming French version. This feature is still experimental and not fully accessible, but a preview of the things to come.


#3

Thanks for the detailed answer!
So a proper date insertion would be yyyy-mm-dd but not yyyy/mm/dd, and with the upcoming extension, all will be considered as date?
Best


#4

Yes, YYYY-MM-DD would be recognized as a date, but not YYYY/MM/DD. Because the parser accepts the extended format you can also input partial dates YYYY or YYYY-MM, approximate dates, intervals, or ranges. Note that some of these (but not all) would currently be formatted nicely. For example this:

Should be displayed as:

If the field in question is marked as a tropy:date field in your template and if you edit the field in the metadata panel (not the item table currently) – it’s still very experimental like I said.

As for migration paths when we roll out the feature properly, you raise a good point: we will automatically migrate valid EDTF values so it’s great if you use EDTF already (it also has the benefit that the sorting as strings is not too bad). We may also provide a way to migrate other values, but haven’t really discussed that yet.


#5

I’m having a related issue, where dates entered in the same format (YYYY-MM-DD), with the same template, and in the metadata panel – but for different photos – are responding differently. Some are remaining in the inputted format, while others are automatically displaying as Jan. 2, 1952, etc. They sort fine so it isn’t a big deal, but is there an easy way to make them follow the same format?


#6

Yes, unfortunately this can happen because Tropy doesn’t fully support datatypes other than strings yet (the date handling isn’t complete yet). For the default templates, you will create ‘date’ values if you originally enter the value in the metadata panel. Currently, if you enter dates (for the first time) via the item table they will always be saved as strings. So my guess is that you entered a few dates via the table. I’m afraid there is no easy way, for now, to convert the datatype via the UI. If only a few dates don’t show up correctly, there are two ways to achieve this:

Clear the date field completely; then restart Tropy (or reload if you have enabled developer mode). After the restart, enter the date again (via the metadata panel). It should now have the proper date type and be formatted according to your locale.

Alternatively, you can select an item where the date is already formatted correctly; then select additional items (using cmd/ctrl click) where the dates are not formatted correctly. Then enter a date in the metadata panel, effectively changing the date values of all the selected items. Obviously this will set the same date value on all items, so you will likely then have to go through all the items again and set the correct dates individually.

We realize that this isn’t an ideal solution and we’ll add better ways to change datatypes once Tropy fully supports more than string values.


#7

Thanks so much for this reply. I fortunately didn’t have this problem with too many so can fix it with the methods you suggest–they are working fine.


#8

I tried to input dates according to the YYY-MM-DD format but it doesn’t display correctly afterwards, it stays in the format of my input:

Can you advise?


#9

This is probably because the Date field has an xsd:string datatype in your template. To enable the partial date value support, you will have to set the datatype to tropy:date.

Unfortunately there is not, yet, an easy way to convert the datatype of existing values (the change in the template only affects the default type for new values), except those I outlined above.


#10

I have the template configured with the datatype tropy:date

and it sill shows up as I enter it (1982-08-09).

And then, while checking things in the template, trying to input the date again, I ended up having a full empty black screen:

This happened with both projects I’m working on at the moment – the very large one of which I emailed you the file yesterday, and another one that I’ve just begun.


#11

Like I said, just changing the template will not automatically convert the existing values (so you would have to clear a field, restart and then enter a new value for the new datatype to be picked up). We’ll improve this as we add proper datatype support.

As for the crash: does that happen every time now? Could you save the project.log file when the crash happens again and post it here?


#12

Sorry I didn’t get it wouldn’t automatically work. After restarting Tropy, it worked, but the date doesn’t appear at all in the main central panel listing all items, it only displays in the metadata panel. I think this is what you were referring to in this conversation earlier?

I email you the project log as it just crashed again.


#13

Just a quick follow-up to note that we fixed the Date/Time entry via the item table in Tropy 1.4.2 and later.